TL;DR: How is the generator expression $<TARGET_RUNTIME_DLLS:…> able to (recursively?) iterate over all linked targets at generation time and can I replicate the behaviour for Linux’ shared objects somehow?
Project background
I am working on a medium sized C++ project in cmake. The project is closed source and consists of a few core shared libraries with their dependencies, Unit and Integration tests with additional test framework dependencies, a collection of samples with additional GUI- and misc. dependencies and a self-built licensing tool.
All of this is cmake based, as we release for multiple Linux distributions and on Windows with Release and Debug configurations on all platforms. Our build is modular and I try to keep up with modern cmake styles, so everything is target based.
For the (admittedly quite complex) dependency management we use Conan and it’s cmake integration with CMakeDeps and CMakeToolchain. With our own conanfile.py recipe we are able to turn off certain parts of the project from Conan, so that the required dependencies are no longer built/checked out (e.g. do not install gtest, if Testing is disabled).
We also use CPack to create shippable packages/installers for our releases.
Dependencies
As already mentioned, most of our dependencies are open source and handled by Conan or at least come with their own cmake package config, so they all have their own targets. Most are thankfully built as static libs, but a few require us to build them as shared libraries. You can probably already see the problems arising.
Dependencies that supply their own package config work fine, they define a SHARED IMPORTED target for themselves with the correct paths written plainly in their IMPORTED_LOCATION or IMPORTED_LOCATION_ (amongst other properties).
The conan built packages are a bit more complicated. They define (depending on the library) multiple internal targets per lib as SHARED IMPORTED, with internal, platform and configuration specific names and then define the “official”/“public” INTERFACE IMPORTED targets (e.g. log4cplus::log4cplus) that links against the correct internal target(s) The problem is, that all relevant properties (INTERFACE_LINK_LIBRARIES, IMPORTED_LOCATION, etc.) use generator expressions to determine the correct target/filepath (e.g. the target qt::qt is an interface with the INTERFACE_LINK_LIBRARIES linking it (amongst others) to Qt5::Core, which is also an interface with INTERFACE_LINK_LIBRARIES of $<$<CONFIG:Debug>:>;$<$<CONFIG:Debug>:CONAN_LIB::qt_Qt5_Core_Qt5Cored_DEBUG>;$<$<AND:$<STREQUAL:$<TARGET_PROPERTY:TYPE>,EXECUTABLE>,$<BOOL:$<TARGET_PROPERTY:WIN32_EXECUTABLE>>,$<NOT:$<BOOL:$<TARGET_PROPERTY:Qt5_NO_LINK_QTMAIN>>>,$<TARGET_POLICY:CMP0020>>:Qt5::WinMain> in x64 Windows, but may link against some other generator expression on other platforms), so they are not known at configuration time
Dependencies in Runtime output folder
For our daily work with the project we need to run our built executable (e.g. Unit Tests) from the build output directory using a debugger. Thankfully on linux the built executable know where the shared objects are that the linker linked them with, but on Windows we need the dll in a path environment (in the case of system libraries, that is fine) or copied to the runtime output directory. A post-build command using the $<TARGET_RUNTIME_DLLS:…> genex is able to resolve all transitive dependency targets through some kind of magic, even if it goes through multiple INTERFACE IMPORTED targets that then determine the correct dll to copy to the output folder.
Dependencies in Relocatable Test packages
For some of our Test, we have to copy the built output and run it on a different machine that may have some test software installed while other development tools are deliberately not installed.
For Windows, we can just copy the runtime build output folder, as it contains all needed dlls already, but on linux the linked 3rd party libs are obviously missing. We already set the RPATH of our executables (build to bin) to “…/lib” so it will look for all missing shared objects in the install folders own lib first. However, for Linux there is no “$<TARGET_RUNTIME_SO:…>” generator expression or similar.
How can I copy my targets transitive dependencies of shared libraries to my CMAKE_LIBRARY_OUTPUT directory in a post build step on Linux?
In a first naive approach I wrote a cmake function that would walk through all (INTERFACE_)LINK_LIBRARIES of my target and, depending on their TYPE, IMPORTED attribute and properties recursively go through the dependency tree returning all (IMPORTED_)LOCATIONs that could then be copied to the runtime/library folder.
However, this obviously breaks if any of these properties use generator expressions, which they sadly do. I am not aware of any way to achieve the same “get-all-transitive-dependencies” using only generator expressions, as there are no “while/foreach/call_function” structures in generator expressions
Dependencies in Install step
Because we use CPack with components to pack our build artifacts into installable packages for different platforms (deb & rpm packages, zip & tar.gz archives and NSIS installer) and CPack uses cmake install steps to create these packages, I also need to specify these dependencies in one or multiple additional install steps per target. The problem here is similar to the relocatable test packages: I can use $<TARGET_RUNTIME_DLLS:…> on Windows to pack just dependencies known to cmake which is exactly what I want, but I have no way to replicate a similar dependency resolution on Linux.
Not working, already tried
Apart from trying to create my own monster function to traverse the dependency tree of my targets during cmakes configuration time (which failes due to generator expressions used on target properties) I also tried a few other things:
- file(GET_RUNTIME_DEPENDENCIES …): returns a billion system libraries from all PATH locations for all possible compilation settings resulting in gigabytes of unwanted libs, that are present on every machine anyway. I could filter these on Windows (as system libs all either start with “api-ms-”, “ext-ms-” or contain “system32” somewhere in the name), but I do not feel qualified to create filters for linux, where system libraries might be different on every system/distribution.
- install(IMPORTED_RUNTIME_ARTIFACTS …): Needs to know the exact SHARED IMPORTED library targets at configuration time and cannot determine transitive dependencies
- install(TARGETS <targets…> RUNTIME_DEPENDENCIES …): same as 1., also applies to RUNTIME_DEPENDENCY_SETs.
- BundleUtilities’ fixup_bundle(…): Does only work with executables, also includes system libs and seems to be intended for MacOS (?). Could not get this to work properly.
- include(InstallRequiredSystemLibraries): As the name suggests, this includes only the generic system libs.
- using $<GENEX_EVAL:…>: Obviously unable to resolve expressions on configuration time, but maybe there is a way to use this for some kind of recursion/loop…?
- install(SCRIPT …) on a file(INSTALL …) inside a script created with file(GENERATE …): Maybe there is gold somewhere in this approach, but I was unable to resolve situations with varying amounts of intermediate interface libraries.
Any help would be greatly appreciated.