How to not install header-sets of private dependencies?

I modernized my CMakeLists.txt by using generated Projectconfig.cmake-files generated by CMake (exported targets) and, especially by using FILE_SET for public and private HEADERS. Works very nicely.

The only thing I have not yet found out is how to instruct the install call to not install the header-files of a PRIVATE dependency of my main product.

add_library(a ...)
target_sources(a PUBLIC ... HEADERS ... a.h)

add_library(lib1 ....)
target_sources(lib1 PUBLIC ... HEADERS ... lib1.h)
target_link_libraries(lib1 PRIVATE a)

install(TARGETS lib1
        EXPORT Lib1Targets
        FILE_SET headers DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})

Here, I’m getting the error, that the dependency from lib1 to a is not in the export-set.
When doing

install(TARGETS lib1 a
        EXPORT Lib1Targets
        FILE_SET headers DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})

I’m getting the public header file of a installed, which I don’t want.

Ideally I’d like to have the dependencies of lib1 installed/exported automatically while respecting the link-scope (no header files installed if private linkage).

Your install(TARGETS) command is asking for the file set to be installed. If you don’t want the file set for a to be installed, but you do want the file set for lib1, and you’ve used the same file set name for both, then you’ll need two separate calls to install(TARGETS) to achieve what you want. You also don’t need to specify the DESTINATION for the file set in your case, since the destination you are specifying is the default anyway.

install(TARGETS lib1
        EXPORT Lib1Targets
        FILE_SET headers
)
install(TARGETS a
        EXPORT Lib1Targets
)

If I’m doing install without the FILE_SET on a cmake bails out with

  install TARGETS target a is exported but not all of its interface file sets are installed

Hmm, yeah I see why that’s the case. It’s a bit of a case of conflicting requirements. If a is installed, then any consumer should reasonably be able to assume it is a complete install. But what you want to achieve is a partial install. CMake is complaining because you’re trying to install a in a way that any arbitrary consumer of it would find parts missing (no headers). It just so happens that you don’t care about those missing parts in your case. I don’t know if there’s a way to convince CMake to do what you want.

I was facing another problem later on: indirect linking with dependent dynamic libraries when RUNPATH is used (and not RPATH). One needs to add manually -Wl,--disable-new-dtags to make modern linkers use RPATH and thus the loader will not clear the paths when loading indirect dependencies.

This seems to be related to my problem: the libraries I don’t want to be installed with their public-headers are the exact ones which I want the loader to load indirectly even though RPATH is not set.

Maybe there is something CMake could abstract: when using CMake super build-project indirect dependencies of shared libraries are common.

Maybe something like:

install(TARGETS lib1
        EXPORT Lib1Targets
        FILE_SET headers
        INSTALL_INDIRECT_DEPENDENCIES

This would only install public headers and, when used via the generated Targets.cmake would add the -Wl,--disable-new-dtags when linking the executable.

EDIT:

I just tried to use RUNTIME_DEPENDENCIES LIBRARY in my install directive, but it doesn’t seems to work. The install(EXPORT ...) call moans about missing dependencies.

I just posted this reply in another thread. RPATH should not be preferred over RUNPATH. The latter is actually more correct, but it exposes insufficiently specified dependencies, which I think you’re trying to do deliberately to work around the “I don’t want to install the headers” problem. If you’re using a superbuild, that really complicates things from an install perspective. Frankly, it’s pretty fragile to try to install things that have been provided to your main build using ExternalProject. It also doesn’t play nice with package managers, if people building your project may want to do that (I don’t know if that’s relevant to your scenario).

What you are saying in the other post is that on libraries which later have indirect dependencies which are not found when only RUNPATH is set on the executable, should have their own RUNPATH set after installing?

In my example I should set RUNPATH of lib1?

Yes, you need lib1 to have its own RUNPATH. If lib1 is coming from an ExternalProject, and you’re using the installed lib1 from that, then in the ExternalProject, you should be setting the INSTALL_RPATH property of that target. That’s most easily done by setting the CMAKE_INSTALL_RPATH variable early in that project before the lib1 target is defined. Probably it should be set to $ORIGIN (at least). I typically use logic similar to the following for a more complete setting:

include(GNUInstallDirs)
file(RELATIVE_PATH relDir
    ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_BINDIR}
    ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}
)
set(CMAKE_INSTALL_RPATH $ORIGIN $ORIGIN/${relDir})