sxs manifest doesn't take effect after first time

Question

My application (myapp.exe) needs to use 2 COM objects that can't be loaded at the same time - they have conflicting dependencies in their manifests. So, my plan:

1. Keep 2 additional copies of the manifest:
   • myapp.exe.manifest.comp1 - references comp1.manifest
   • myapp.exe.manifest.comp2 - references comp2.manifest
2. Before I run myapp.exe, copy the desired myapp.exe.manifest.comp1/2 to overwrite myapp.exe.manifest.

However, I found that Windows still uses the original myapp.exe.manifest, see Test below. It seems Windows keep some kind manifest cache somewhere?

Question: How can I replace manifests, and get the replaced manifests to take effect?

Test

1. Copy myapp.exe.manifest.comp1 myapp.exe.manifest
2. Run myapp.exe
   • Result: myapp.exe.manifest.comp1 takes effect, and comp1 works
3. Close myapp.exe, and Copy /y myapp.exe.manifest.comp2 myapp.exe.manifest
4. Run myapp.exe
   • Expected: myapp.exe.manifest.comp2 takes effect, and comp2 works
   • Actual result: myapp.exe.manifest.comp1 still takes effect, and so comp2 doesn't work

Investigation:

• Looking in procmon, I see that when I run myapp.exe, some other processes (the launching powershell.exe, and also csrss.exe) do access myapp.exe.manifest. So it's not completely ignored...
• Using sxstrace, I see that myapp.exe.manifest is getting used. But I see afterwards that Windows only searches in the paths taken from myapp.exe.manifest.comp1

Edit: An alternative would be to:

• Load manifests in runtime using Activation Context APIs - ActivateActCtx etc., but so far I haven't gotten that to work...
• From comment: For unmanaged code, use SetDllDirectory/AddDllDirectory
• For managed code, use app.config and/or AssemblyResolve