Do the files end up clobbering each other in the package? That is, do the release and debug libraries both have names like libfoo.so.1 and the release configuration just gets copied afterwards and therefore “wins”?
Other than that, what generator are you using for the build itself?
Under the build_win64/._CPack_Packages/win64/NSIS64/ there seems to be a naming convention where DLLNAMEHERE.dll (the release file) has a sister file libDLLNAMEHERE.dll (the debug file).
So I was building to same directory as release, I think I should build under different directory build_win64.debug and try that way.
I skipped the NSIS installer and manually copied the debug binary files to a another machine, I believe I now have a proper installation to work with.