Control what is installed / packaged

Code
1

Hello,

I’ve writtem some install logic and cpack logic for my application. It generally works, but it literally installs everything.

I use FetchContent whenever possible so all my dependencies are subdirectories. If I install my application everything from these dependencies will get installed as well including documentation, header files, cmake scripts and other stuff.

How can I just install my application and everything that is necessary for it from the dependencies. But exclude all not needed things like header files or documentation?

Thanks!

2

The subprojects would need to offer options for you to disable their install rules or, if it works, you can use install components to install just what you need. I don’t think CMake has a way to say “ignore install rules for this directory”, but maybe some custom install code that calls return () before all of the subdirectories run could work?

2 Likes
3

Hmm my own install target has a component, so I could just install that target. But then shared libraries are missing for instance.

I though maybe using “RUNTIME_DEPENDENCIES” would help with this, but for some reasons it doesn’t.

install(TARGETS BeansApp COMPONENT Beans 
  RUNTIME_DEPENDENCIES
    PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
    POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
    DIRECTORIES $<TARGET_FILE_DIR:Qt6::Core>
    FRAMEWORK DESTINATION Beans.app/Contents/Frameworks
)

Shouldn’t this be able to find my shared dependencies? Or at least try to find them. It searches for Qt6 and also installs that sucessfully, but it doesn’t search for all linked libraries, which came via FetchContent. At least they are not installed neither is there an error message.

Directory based Ignoring, no matter if build in or manual, would probably not helper either. The libraries often have all their install stuff in one directory. And they also don’t use Components.

I suppose the best way would be if they all use Components, but it would be insane to open Pull Requests for all the dependencies. Furthermore the Component name is free to choose, so even if they all specify components, it would be quite some work to figure out what components are needed exactly.

Couldn’t cmake provide an option to automatically install shared target dependencies. So for example look for all shared targets and then only execute install(target …) commands found for these targets and ignore everything else?

4

Actually I had a look again at runtime dependencies and the issue seems to be the RUNTIME_DEPENDENCIES of install. The manual approach works:

install(CODE [[
  file(GET_RUNTIME_DEPENDENCIES
    LIBRARIES "$<TARGET_FILE:BeansApp>"
    RESOLVED_DEPENDENCIES_VAR _r_deps
    UNRESOLVED_DEPENDENCIES_VAR _u_deps
    DIRECTORIES "$<TARGET_FILE_DIR:Qt6::Core>"
    PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
    POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
  )
  foreach(_file ${_r_deps})
    file(INSTALL
      DESTINATION "${CMAKE_INSTALL_PREFIX}/bin"
      TYPE SHARED_LIBRARY
      FILES "${_file}"
    )
  endforeach()
  list(LENGTH _u_deps _u_length)
  if("${_u_length}" GREATER 0)
    message(WARNING "Unresolved dependencies detected!")
  endif()
]] COMPONENT Beans)

Here is a small test with a comparison:

image
image1031×813 95.7 KB

As can be seen the install commands puts the third party library I tested here in a “POST_EXCLUDE_FILES_STRICT”. Is this intentional?

5

I would guess that it is skipping recursive scanning to avoid double installation. You could add your own install rules for targets in the subdirectories so that you can bypass whatever other rules are there.

6

I appologize for the dig, but @Leon0402 I was wondering if in the end you just ended up sticking with the file(GET_RUNTIME_DEPENDENCIES approach in the end.

I’m in a similar situation where I use FetchContent as much as possible and thus simply installing the targets within the build tree would result in everything pouring in to the install directory, most of which I don’t want. I already use a technique in which any projects that are not the top-level project automatically have EXCLUDE_FROM_ALL appended to their calls to install(...), which has worked great for static builds, but now for shared builds I need to somehow specifically allow the runtime dependencies through.

I can of course do this by making the exclude statement not apply to the RUNTIME category, but then I’d also need to add a switch to AND that against as in my lib only projects I still wouldn’t want the runtimes to be installed, only on projects that feature executables. Then finally, the biggest flaw as you pointed out is that this only works for my own projects and doesn’t apply to other’s that don’t follow the exact same design pattern (i.e. 0), unless you get lucky and can bodge something together using FetchContent arguments and creating custom imported targets/installs, but that still is pretty crude.

I find it more idiomatic and desirable to have the install of the runtimes “pulled” by an executable itself rather than to “just install the library”, in the context of the library being a subproject. In that context, I essentially don’t care about the library’s install contents and only want what the executable needs, so in these projects I never actually install the library directly, not through the default install component nor the named one I made for it. It’s the executable that necessitates the installation of the some of the library’s components, not the library itself.

install(TARGET ... RUNTIME_DEPENDENCIES) seemed perfect until I learned that to my chagrin it forcefully excludes in-tree targets as you noted. IMO there should just be a switch argument for that.

It seems to me the only real downside of included in-tree targets is that if you do happen to install a library directly (maybe one that’s part of the top-level project) then any of it’s runtimes that an executable is dependent on will cause them to them be queued for installation twice. While I get that in really large and complex projects this can add a siginificant time waste, for smaller or even medium sized ones this just means a few extra potential “Up-to-date” messages in the console with an insignificant extra amount of time spent, especially in my case where I guarantee that sub-project targets are not installed as part of ‘all’.

Personally I’d much rather take that trade-off than the mass of changes I’d have to perform on my own projects and hackery I’d have to attempt on other’s when fetching them.

So basically, I’m just curious if this approach ended up working out for you.

7

I use a combination of automatic and manual approach:

install(TARGETS BeansClient COMPONENT Beans 
  RUNTIME_DEPENDENCIES
    PRE_EXCLUDE_REGEXES "api-ms-" "ext-ms-"
    POST_EXCLUDE_REGEXES ".*system32/.*\\.dll"
    DIRECTORIES $<TARGET_FILE_DIR:Qt6::Core>
    FRAMEWORK DESTINATION Beans.app/Contents/Frameworks
)

foreach(lib IN ITEMS CLI11::CLI11 sqlpp11::sqlite3 spdlog::spdlog)
  get_target_property(target_type ${lib} TYPE) 
  if (target_type STREQUAL SHARED_LIBRARY)
    get_target_property(real_lib ${lib} ALIASED_TARGET)
    install(TARGETS ${real_lib} COMPONENT Beans)
  endif ()
endforeach()

Basically I install all my fetch content targets myself. So it’s not comletly automatic as I have to list the dependencies, but it works quite well. Perhaps there is even a solution to automate this, but it wasn’t worth the effort for me.

1 Like
8

And for Linux you may want to set:

POST_EXCLUDE_REGEXES "^/lib" "^/usr/lib"

to skip any system libraries

9

Yes, that’s a good call and matches blocking system32 on Windows obviously.

This is my general purpose process that I’ve thrown together and use by default:

if(WIN32)
     set(_path_runtime "${CMAKE_INSTALL_BINDIR}")
else()
     set(_path_runtime "${CMAKE_INSTALL_LIBDIR}")
endif()

install(CODE "set(_EXECUTABLE \"$<TARGET_FILE:${_TARGET_NAME}>\")"
    COMPONENT ${_TARGET_NAME}
)
install(CODE "set(_RUNTIME_PATH \"${_path_runtime }\")"
    COMPONENT ${_TARGET_NAME}
)
install(CODE [==[
    file(GET_RUNTIME_DEPENDENCIES
        EXECUTABLES "${_EXECUTABLE}"
        RESOLVED_DEPENDENCIES_VAR _runtime_deps_resolved
        UNRESOLVED_DEPENDENCIES_VAR _runtime_deps_unresolved
        PRE_EXCLUDE_REGEXES
            [=[api-ms-]=] # VC Redistibutable DLLs
            [=[ext-ms-]=] # Windows extension DLLs
            [=[[Qq]t[0-9]+[^\\/]*\.dll]=] # Qt Libs, don't block on Linux since users likely only have older Qt available
        POST_EXCLUDE_REGEXES
            [=[.*system32\/.*\.dll]=] # Windows system DLLs
            [=[^\/(lib|usr\/lib|usr\/local\/lib)]=] # Unix system libraries
    )
    if(_runtime_deps_unresolved)
        foreach(_udep ${_runtime_deps_unresolved})
            message(SEND_ERROR "Failed to resolve dependency: ${_udep}")
        endforeach()
        message(FATAL_ERROR "Unable to resolve all dependencies for executable ${_EXECUTABLE }")
    endif()

    foreach(_rdep ${_runtime_deps_resolved})
        file(INSTALL
          DESTINATION "${CMAKE_INSTALL_PREFIX}/${_RUNTIME_PATH}"
          TYPE SHARED_LIBRARY
          FILES "${_rdep}"
          FOLLOW_SYMLINK_CHAIN
    )
    endforeach()
    ]==]
    COMPONENT ${_TARGET_NAME}
)

I handle Qt DLLs using its macros in a subsequent step and as the comment above suggests I tend to use a lot of custom builds of Qt with newer versions that aren’t available on some distros, so I let the shared libraries for those go through on non-Windows platforms.