C –depends-on→ shared-lib-B –depends-on→ shared-lib-A
B doesn’t expose any parts of A to C; so, B’s dependency on A is PRIVATE. When C goes to link with B at build time, there are no symbols from A to resolve and the linker is happy.
But A and B end up being installed to different directories. When running C, its INTERFACE_LINK_DIRECTORIES only includes B’s installation directory because B’s dependency on A is PRIVATE, AFAICT. So, the run-time linker cannot find A and C fails to run.
Should A’s install directory be included in B’s INTERFACE_LINK_DIRECTORIES even though A is a PRIVATE dependency?
I suppose the counterargument to this is: INTERFACE_LINK_DIRECTORIES is only concerned with what’s needed to resolve things at build-time. If that’s the case, what, then, should clients be using to derive a correct LD_LIBRARY_PATH to run C?
On non-Windows systems, clients shouldn’t be needing to set LD_LIBRARY_PATH or similar at all. Set up your RPATH details and rely on those instead. That is far more reliable and convenient for consumers, as long as all your dependencies also set up their RPATH correctly (which third party vendors often don’t, unfortunately).
I don’t have a solution for you for Windows, sorry. It’s lack of RPATH support is a constant PITA.
Historically I’ve tended to avoid RPATH due to concerns with relocatability of binaries as well as its effect on expected behavior when LD_LIBRARY_PATH is used. I’m willing to entertain the notion that these concerns are outdated or otherwise misinformed.
My current use case is Conan, where each package is effectively “installed” under its own prefix. (And, yes, I’m well aware of the Virtual[Build|Run]Environment stuff; but it comes with its own set of inconveniences that I am hoping to avoid by solving this at the CMake level.) I have gotten rather far by inspecting INTERFACE_LINK_DIRECTORIES at the CMake level; but it does require that PRIVATE shared library dependencies be explicitly accounted for in the consumer’s INTERFACE_LINK_DIRECTORIES. Am I correct that the RPATH solution would involve a similar explicit propagation of a PRIVATE shared library dependency’s INTERFACE_LINK_DIRECTORIES to the consumer’s INSTALL_RPATH?
Old RPATH doesn’t play nice with environment variables like LD_LIBRARY_PATH. New RUNPATH does play nice, and my understanding is that most linkers will now embed RUNPATH by default instead of RPATH. You can explicitly state which of the two behaviors you want with linker flags like --enable-new-dtags, but these days, that’s unlikely to be necessary. I talked a bit about the RPATH / RUNPATH differences in my 2019 CppCon talk, I think somewhere in the back half of that talk.
The Conan case is more interesting because, as you observed, each package is installed in its own separate directory. That means the relative path to other dependencies no longer follows the more standard layout that you’d get if they were collocated like for packages distributed as part of the OS. I don’t know how Conan manages the RPATH details of its packages, but I haven’t come across anything in the various recipes and generated files that seem to do anything special (but I’ve been working mostly with static libraries via Conan). I can’t comment on your INTERFACE_LINK_DIRECTORIES approach, I can’t say I’ve ever used that directly myself. I don’t know if the INSTALL_RPATH_USE_LINK_PATH target property and its associated CMAKE_INSTALL_RPATH_USE_LINK_PATH variable might be closer to what you’re looking for or not.
Another advantage or the RUNPATH is its natural ability to allow users to replace the versions of the libraries by their own versions by setting LD_LIBRARY_PATH. Since the ability to do this is a requirement of some open-source licenses (some LGPL ones I think?), if your library has such a license then building your libraries with RUNPATH would directly support e.g. closed-source consumers.
)