Copying created shared libraries

Code
1

My project has 3 shared libraries, and 2 executables.

During the Install phase, everything is copied to the correct location. However when debugging, the shared libraries must be copied into the directory where the executable being debugged is created.

This is easy enough to do using add_custom_command( TARGET exe POST_BUILD <COPY_CMD>)

Where the copying of the file command is <COPY_CMD>

However, I need this function to be run not only when the executable is linked, but when the shared library is built as well. Otherwise the executable can be run on the old shared library.

Ive tried the add_custom_command for the shared library, because the

The copy command, in general is something like
${CMAKE_COMMAND} -e copy srcfile $<TARGET_FILE_DIR:executable>

which leaves me to the following problem

CMake Error at (add_custom_command):
TARGET ‘executable’ was not created in this directory.

Any ideas would be most appreciated.

2

A common alternative solution to Windows’ lack of something like RPATH is to set the output directory of all executables and shared libraries to the same location. It isn’t always feasible (and personally I dislike it, since the clutter is considerable for large projects), but if it suits your situation, that is probably the simplest way to get easily debuggable binaries. Projects taking this approach typically set CMAKE_RUNTIME_OUTPUT_DIRECTORY sufficiently high in their directory structure to capture all the executables and shared libraries the project defines. You would also need to check how this might affect dependencies also built by the project. Another wrinkle is test executables - sometimes there could be very many of these, and you might not want them mixed in with the non-test executables.

I generally discourage copying around DLLs in POST_BUILD rules. You end up with multiple copies of your DLLs littered around your build directory and it’s easy to end up using an out-of-date DLL. As you’ve discovered, it isn’t always easy to get the DLLs you want copied, and it gets combinatorially worse the more executables you have that need your DLLs. Furthermore, you may also need to copy around PDBs (I don’t recall if that’s needed or not). Given the downsides, if you can avoid copying DLLs around like this, that would be better.

3

Unfortunately, both situations are poor.

What I used to do, and was looking for a better solution. is add to the executable debug paths, the path to the built shared library directories.

4

In 3.21 they added $<TARGET_RUNTIME_DLLS:tgt>

List of DLLs that the target depends on at runtime. This is determined by the locations of all the SHARED and MODULE targets in the target’s transitive dependencies. Using this generator expression on targets other than executables, SHARED libraries, and MODULE libraries is an error. On non-DLL platforms, it evaluates to an empty string.

This generator expression can be used to copy all of the DLLs that a target depends on into its output directory in a POST_BUILD custom command. For example:

find_package(foo CONFIG REQUIRED) # package generated by install(EXPORT)

add_executable(exe main.c)
target_link_libraries(exe PRIVATE foo::foo foo::bar)
add_custom_command(TARGET exe POST_BUILD
  COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:exe> $<TARGET_FILE_DIR:exe>
  COMMAND_EXPAND_LISTS
  )

Note

Imported Targets are supported only if they know the location of their .dll files. An imported SHARED or MODULE library must have IMPORTED_LOCATION set to its .dll file. See the add_library imported libraries section for details. Many Find Modules produce imported targets with the UNKNOWN type and therefore will be ignored.

1 Like
5

THANK YOU… I will look into that variable.

Do you know if the variable is modifiable? or only determinable by the system.

Ill have to investigate how “runtime” is determined in "depends on at runtime. My bet its defined as statically bound shared libraries, that the executable requires to launch properly. Which means I may need to tweak it a bit, because I have a couple of “dynamically bound when needed by the executable” runtime dependencies (plug ins)

But thanks again!

6

Unfortunately, this new TARGET_RUNTIME_DLLS doesnt help copy when the DLLs change, only when the main exe changes

7

That’s pretty surprising to hear. Maybe you could create a SSCCE?

Are you linking the libraries using target_link_libraries? How are you linking against the DLLs in question.

8

In this project I have 4 Shared libraries.

2 of them are dynamically bound, dynamically loaded image formats (One is a viewer for BIF/Roku index images for Qt, the other is a Multi-Frame GIF writer for Qt as well)

The main application, doesnt even know they “exists” until the Qt image system loads them up. But they have to be placed in the correct location, and updated if they change.

One of the remaining two, is statically bound dynamically loaded. When the exe is built, its easy to add a custom command (and now using the TARGET_RUNTIME_DLLS variable) to copy those files into the correct location for debugging.

The final dll, is statically bound inside the previous dll. So I am working to use TARGET_RUNTIME_DLLS on the other DLL to get this DLL for copy.

However, what I cant figure out, is what is the best way to copy the DLLs (all 4) when any of them is built.

Meaning, I tweaked my image processing DLL, and I need to copy it into the image format directory. Nothing in its API changed, so there were no triggers to rebuild the main executable…

However,

9

The only good solutions are either RUNTIME_OUTPUT_DIRECTORY target property or a qt.conf (but IIRC you can only specify one directory for each setting).

Why do you insist on copying?

10

Sounds like you already know where the libraries location when they’re built.
There are properties for each target that track the location.

You could gather all that info and set VS_DEBUGGER_ENVIRONMENT or a batch file that could export the updated path.

It’s not like it would change often and you wouldn’t have to copy DLLs every time they get updated.

But I’ve always set CMAKE_RUNTIME_OUTPUT_DIRECTORY on Windows.

11

I like the idea of the VS_DEBUGGER_ENVIRONMENT, I didnt realize that got added.

The RUNTIME output dir, would on make a difference for the install stage.

12

I set VS_DEBUGGER_ENVIRONMENT only for external libraries like Qt.

The install rules for targets always take the right file positions, setting RUNTIME_OUTPUT_DIRECTORY or not.

13

The problem with the external libraries for Qt, is they have to be in the correct location. For me, since two are image formats, in the “executable/imageformats” directory

So VS_DEBUGGER_ENVIRONMENT for PATH alone wont work, I forget if Qt has another env var I can set for find plugins

14

If you have the Qt core library in PATH environment variable, it will find plugins based on its own path.

You could check if qt.conf file can help you with this.

15

I was trying to avoid using qt.conf. But I am able to use
QT_PLUGIN_PATH however it looks for DIR/imageformats/XXX.dll not DIR/XXX.dll

So now Im trying to figure out why the LIBRARY_OUTPUT_DIRECTORY on a given shared library target isnt working :frowning:

16

On Windows, libraries are split into a .lib for the linker and a runtime part, the .dll. These can go into two different directories.

17

On Windows, libraries are split into a .lib for the linker and a runtime part, the .dll. These can go into two different directories.

The problem is, it always adds Debug/Release etc after the OUTPUT_DIRECTORY

I need to set both LIBRARY_OUTPUT_DIRECT and RUNTIME_OUTPUT_DIRECTORY if I want the .lib, .pdb and .dll in the same directory. Once I realized the DLL/PDB are runtime targets I was able to set this up.

Unfortunately, due to linux limitations, you can not set the OUTPUT_NAME to …/libname

Unfortunately I think Im stuck copying the files

18

Yeah! i got it figured out.

To create the plugin based directory structure use the following

set_target_properties( imagelib PROPERTIES 
                                RUNTIME_OUTPUT_DIRECTORY "$<CONFIG>/imageformats"
                                ARCHIVE_OUTPUT_DIRECTORY "$<CONFIG>/imageformats"
                                LIBRARY_OUTPUT_DIRECTORY "$<CONFIG>/imageformats"
)

Then for the executable, you set the following

set_target_properties( exec 
                                    VS_DEBUGGER_WORKING_DIRECTORY "$<TARGET_FILE_DIR:exec>" 
                                    VS_DEBUGGER_COMMAND "$<TARGET_FILE:exec>" 
                                    VS_DEBUGGER_ENVIRONMENT "PATH=PATH=%PATH%;$<TARGET_FILE_DIR:sharedlib1>;$<TARGET_FILE_DIR:sharedlib2>\nQT_PLUGIN_PATH="$<TARGET_FILE_DIR:plugin1>/..;$<TARGET_FILE_DIR:plugin2>/.."
                     )

Note 1, in the VS_DEBUGGER_ENVIRONMENT you must separate the variables by an explicit \n for Visual Studio to recognize multiple variables being set.

Note 2, in the QT_PLUGIN_PATH you are setting it to $<TARGET_FILE_DIR:plugin1>/… Note the /…, since the target file dir includes imageformats

I have not tested this on other IDE systems, so YMMV on xcode etc

Thanks for tall the help

1 Like