Clarification: Shared object (DLL) copying

Code
1

Over the decades, a lot has been written about associating a DLL with an executable and getting that DLL into install directories, or getting that DLL into ctest directories.

I’m trying to modernize a 14 year old CMakeLists based project, and several targets end up needing an out-of-tree built imported.dll (fmod if you need a concrete example), which is linked against through a low-level internal library, that other internal libs link against, and ultimately targets innocently acquire.

wrapper(links dependency) ← lib1(links wrapper) ← lib2(links lib1) ← exe(links lib2)

This was an early component of the projects so … it’s done with includes, variables, and a"post build" function that uses copy_if_different but has to be called by every target that ends up using sound.

EDIT:
I’ve created a small demo repository for this: GitHub - kfsone/cmake-dlldep: Demonstrate how to implement a dll dependency in cmake

What I want to achieve:

  • .dll files automatically copied by down-stream “install()” calls,
  • .dll files automatically copy alongside their .exe in the build tree,
  • no function calls or variable use after the definition of target that links the dll’s .lib,

I imagine the code being something like:

add_library(DependentDLL SHARED IMPORT)
set_target_properties(DependentDLL PROPERTIES ... $<secretsauce> dep.dll)

add_library(lib1 lib1.cpp)
target_link_libraries(lib1 PUBLIC DependentDLL)

add_library(lib2 lib2.cpp)
target_link_libraries(lib2 PUBLIC lib1)

add_executable(program program.cpp)
target_link_libraries(program lib2)
install(TARGETS program DESTINATION ${INSTALL_PATH} COMPONENT program)

So executing:

$ cmake -G Ninja -B ${build_dir} -DCMAKE_PREFIX_PATH=${tmpdir}
$ cmake --build ${build_dir} --target program
$ cmake --install ${build_dir} --component program

must result in the following files:

  • ${build_exe_dir}/program.exe
  • ${build_exe_dir}/dep.dll
  • ${tmpdir}/program.exe
  • ${tmpdir}/dep.dll

Is this something that can be done cleanly, i.e without having to trawl GET_RUNTIME_DEPENDENCIES or invoke ADD_CUSTOM_COMMAND (directly or indirectly) downstream of the actual linkage?

Also, based on the amount of conflicting and confusing stack/quora/etc posts about this it seems something worth documenting with examples in the cmake docs.

1 Like
2

I created an MVCE: GitHub - kfsone/cmake-dlldep: Demonstrate how to implement a dll dependency in cmake

3

Working on the MVCE, I’ve actually managed to get this MUCH closer with the CMake 3.21 imported library features:

To the ‘SHARED IMPORT’ library I added:

INSTALL(
	IMPORTED_RUNTIME_ARTIFACTS

	ExternalDLL

	RUNTIME DESTINATION   "${CMAKE_CURRENT_LIST_DIR}/../Artifacts"
	LIBRARY DESTINATION   "${CMAKE_CURRENT_LIST_DIR}/../Artifacts"
	FRAMEWORK DESTINATION "${CMAKE_CURRENT_LIST_DIR}/../Artifacts"
	BUNDLE DESTINATION    "${CMAKE_CURRENT_LIST_DIR}/../Artifacts"
)

and now I’m getting .dll files copied to the destination, but still no automatic equivalent of doing an add_custom_command(... copy_if_different ... dll.dll ${CMAKE_CURRENT_BINARY_DIR} ...)

4

The underlying command is file(GET_RUNTIME_DEPENDENCIES). You can attach a CMake script with this call to a target’s POST_BUILD to do the copying at that time.

5

From the OP:

i.e without having to trawl GET_RUNTIME_DEPENDENCIES or invoke ADD_CUSTOM_COMMAND (directly or indirectly) downstream of the actual linkage

And, you can nearly do it with CMake 3.21; it looks like SET_TARGET_PROPERTIES doesn’t take generate expressions for IMPORTED_IMPLIB or IMPORTED_LOCATION, and you still need the surprise! post-build step on every downstream target to get the dynamic library copied alongside executables in their $<TARGET_FILE_DIR>.

6

Correction: you can mostly do it with CMake 3.21 with SET_TARGET_PROPERTIES and IMPORTED_LIB, and IMPORTED_LOCATION. These don’t support generator expressions but they do have _<CONFIG> support.

But you’d still need an ADD_CUSTOM_COMMAND build step to ensure the initial executable artifact (approx ${CMAKE_BINARY_DIR}/${OUTPUT_NAME}) has a copy.

7

Yes. Generally, CMake assumes the environment is set up “right”. On Windows, this means that PATH is populated with DLL dependencies. Of course, projects can work to alleviate this, but it’s not easy for CMake to do all of these things. Of note for this case are the following considerations:

  • dependency tracking: Should the files be copied if the DLLs update? Should it rerun if the destination DLLs are deleted?
  • cleaning: Should make clean and ninja -t clean remove these DLLs? How does it know which they are?
  • exclusions: Which DLLs should be excluded (e.g., Windows SDK and system libraries)?

These are questions that specific projects can answer for themselves, but I don’t think CMake can do it automatically without satisfactory answers here.

8

I use a similar approach.

I have a function that recursively goes through the exe-targets dependency tree using the LINK_LIBRARIES, IMPORTED_LINK_INTERFACE_LIBRARIES_config and IMPORTED_LINK_DEPENDENT_LIBRARIES_config properties and collects the LOCATION_config or RUNTIME_OUTPUT_XXX properties to get the paths to the required .dll files. Instead of a POST_BUILD event I use a custom target to copy the files into the build-tree. I let the exe-target then depend on this custom target to make sure files are deployed before the exe is built

This is a lot and ugly code though and it breaks down when properties contain generator expressions. Also it may depend on how external projects populate the properties of their import-targets and break down when that changes. When I wrote that code the file(GET_RUNTIME_DEPENDENCIES) did not exists so maybe I could use that to simplify my setup.

I had to solve this problem for every Windows CMake project I worked with, and depending on my mood and CMake knowledge of that time I solved it slightly differently. For example you could do the dopying at generate time instead of doying it at build-time.

I do not claim that this is the best way of doying it but if you want to get some inspiration or copy some code snippets you can look at my CMake library project CPFCMake on github.

cpfLinkTreeUtilities.cmake → Contains the functions for getting the targets in the link-tree.

cpfAddDeploySharedLibrariesTarget.cmake → Contains the code for creating the custom-dll-copy-target.

If you happen to use Conan as your package manager, you can also consider doing this dll-copying in the conanfile because conan also has the dll-location information.

A solution on CMake side would be very much appreciated to remove that burden from the users.

9

Caveat this I am oblivious to the implementation, I’m not saying “based on knowledge of cmake’s code I think”, I am purely speaking to anecdotal similarity to past experiences and my superficial awareness of “recent” cmake improvements: don’t the changes in 3.21 w/r/t IMPORTED_RUNTIME_ARTIFACTS any opportunities here.

In particular, re exclusions: I got to see exactly what you’re talking about with my experiments, but cmake appeared to DTRT when I configured it as exe <–public-- lib2 <–public-- lib1 <–public-- [shared imported global] externlib (see kfsone/cmake-dlldep: Demonstrate how to implement a dll dependency in cmake (github.com) for a full example, the latest version works on Windows with 3.21 or 3.22, but only the install)

cleaning: Pre-3.21, the end user had to deal with that as well as get the DLLs there in the first place. And, sure, if you were launching the wrong dll because cmake didn’t copy_if_different, that could cause frustration. But, I can’t count all the times I’ve worked up a decent rage trying to get autoconf, cmake or some proprietary build system to put a file in the right place in the first damn place, and I think of all the times I’ve had a build system leave a file when I did a clean, or not have a clean option, and I don’t remember once thinking more than “oh, that’s annoying”. Conversely, I can’t think of a single time I’ve spent 2-3 days wrestling a .dylib or framework or .so or .dll into place and thought “well, it builds character”, hehe :slight_smile:

10

The question is if it builds a desirable character or if it will turn you in an old, bitter and frustrated man :wink:

Although when you use the custom target approach and you get your output- and input-paths right it should do the copying only when necessary.

1 Like