External Annotation for DLL randomly changes

I have created several external annotations files for NuGet packages our project uses, and they work great, but periodically, the external annotations will stop working.  Upon investigating the issue, it appears that ReSharper references the external DLLs from a different location then the NuGet packaget, despite Visual Studio referencing the DLLs from the NuGet package.  This then means that the external annotation files I have created are no longer next to the DLLs according to ReSharper.

Example:

I have 3 projects in my solution: UI Project, DAL Project, and a Unit Test Project.  The UI Project has a reference to the DAL project and the Unit Tests project has references to both the DAL Project and the UI Project.  The UI Project and the DAL Project have a reference to the external DLL I have created annotations for.  This works fine for a while, but eventually, ReSharper will start referencing the external DLL from my Unit Test Project's bin output folder after building, and then the external annotations are lost.  I'm assuming this is where ReSharper is looking for the external annotation as when I do a peek definition on the methods I'm annotating the comment at the top of the generated code has this Unit Test Project/bin folder path, and when it has the NuGet Package path the annotations work.  I can only resolve this by deleting the contents of the Unit Test Project's bin folder, then ReSharper sees the annotations again, until my next build.