How to use libraries conveniently on Windows

Hi everyone,

I’m currently trying to include a lib (soci) in my cmake projects. After days of hacking around, I got it working. But there are still a few problems, which I solve with ways I don’t like :wink:

Maybe some of you have suggestions on how to make that better.

  1. Automatically include headers of libs

So I basically created a (very simple) sociConfig.cmake how it is explained in “Professional cmake” by doing:
install(EXPORT SOCI NAMESPACE SOCI:: DESTINATION cmake FILE SOCIConfig.cmake)

This allow me to link against cmake targets. To automatically include the right headers I did add such a line to the install commands of the targets:
install(TARGETS ... INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})

So that’s actually working fine for that target, but the problem is with dependencies. Soci itself has dependencies to sqllite3. So I don’t have to include the headers of soci anymore, but still the headers of sqllite (in order that the soci headers work). How can I achieve that does headers will be included automatically as well?

  1. Using .dlls on Windows

I have another problem on Windows, which I don’t have on Linux (due to the fact that libs are in a known place I guess). If I build my application it’s missing all the dlls of soci as well as sqlite3. Manually copying them in the directory solves that issue, but that doesn’t seem right to me.

If I say target_link_libraries(someTarget someOtherDynamicLibTarget) , I assumed that cmake would do the job for me and put the dlls in the right place. But it seems like that isn’t the case?
I have read in several places solutions like copying the dlls. via a cmake function … but that doesn’t seem right to me?
After all that’s the purpose of target_link_libraries isn’t it?

What’s the correct (platform independent) solution to make it work conveniently on Windows as well as Linux?

You shouldn’t be writing a SOCIConfig.cmake file, that file should be provided by SOCI itself. SOCI should be taking care of specifying all the relevant dependencies as part of that. I think @mloskot was working on improved CMake support for the 4.0 SOCI release, but I’m not sure what state that got to.

Putting DLLs “in the right place” implies a particular view of how those DLLs should be handled. For linking, CMake should do the right thing and link DLLs without having to copy them around. The target_link_libraries() doesn’t (and shouldn’t) copy DLLs for you, there is no requirement that a DLL be in the same directory as the executable that uses it. It is a different story for running an executable. People often assume that the DLL must be in the same directory as the executable, but that’s just one way for the OS to find the DLL when running that executable. It also searches the PATH. If you want to avoid copying around DLLs (something I generally advise avoiding if you can), you will need to augment your PATH environment variable before launching the executable.

There are a few ways of augmenting your PATH, depending on how you want to be launching the executable. Since you mentioned that you have my book, look in the Project Organization -> Windows-specific Issues section which goes through different approaches. For everyone else, the techniques there talk about:

  • Using the ENVIRONMENT test property to set the PATH environment variable for you (this takes care of invoking executables as part of ctest runs).
  • Using the VS_DEBUGGER_ENVIRONMENT target property (this takes care of invoking executables within the Visual Studio IDE)
  • Writing your own user.props file if you are using an earlier version of CMake that doesn’t have support for VS_DEBUGGER_ENVIRONMENT but does support the VS_USER_PROPS target property.

If you’re looking for a convenient way to invoke an executable outside of those contexts, a common way of handling that is to write your own launch script that augments the PATH before launching the executable. CMake currently doesn’t provide any help to you for this, so you’re on your own with it unfortunately. If you really want to go down the path of copying DLLs next to your binary though, you could try taking a look at using file(GET_RUNTIME_DEPENDENCIES) in conjunction with add_custom_command(TARGET someTarget POST_BUILD ...). That file() subcommand is more intended for use at install time, but I’d assume you could probably get it to work at build time. It might be annoying for developers to have it getting invoked every time the target is built though.

Hi! Thanks for your reply :slight_smile:

You shouldn’t be writing a SOCIConfig.cmake file, that file should be provided by SOCI itself. SOCI should be taking care of specifying all the relevant dependencies as part of that. I think @mloskot was working on improved CMake support for the 4.0 SOCI release, but I’m not sure what state that got to.

I know, I read your book :stuck_out_tongue: I hacked in the source code of soci itself because it seems like mloskot isn’t active anymore. Soci 4.0.0 is already released. I plan to do a pull request as soon as I have something working good. But there is still some work need to be done supporting also components and stuff like that … and I’m not sure if I have understood yet how that correctly works. Might ask another question here regarding that.

People often assume that the DLL must be in the same directory as the executable,

That’s basically my view of Windows yeah. I’m usually only using Linux, but I try to add support for Windows for my application and it seems just hell to me (more because of Windows than cmake I supppose).

Ah I just saw in the chapter that you said that rpath is not supported on Windows. That was my second idea how that should be handled :confused:
Hopefully that will be supported in the future.

These tips are helpful, especially in the context of ctest. Unfortunately, I would like to also support executing the application just in the terminal (without VS).
Would that be worth a feature request? An ENVIRONMENT Property which sets the PATH until set again (so another cmake run?)
Or is there any work in progress to make handling dlls easier (like adding rpath on Windows)?

So it seems like the best option I have is putting the dlls on my path manually if I understood you right? Copying is also possible, but perhaps not the cleanest solution.

Do you also have an idea for my first problem?

Thanks a lot! :slight_smile:

@craig.scott

I had not managed to push the CMake configuration overhaul too far before releasing SOCI 4.0.0

From SOCI 4.0.0 release notes (CHANGES file)

  • Added basic package exporting to CMake configuration (#503).
  • Although the build configuration is based on CMake 2.8,
    numerous improvements have been applied to the CMake scripts.

My initial attempt of the modernisation is dangling in ml/modern-cmake branch in the SOCI repo.

FYI, I have retired as SOCI maintainer and release manager,

@Leon0402 If you are willing to contribute CMake modernisation for SOCI, that is awesome. I’m sure Vadim or whoever currently maintains the library now will appreciate your help.

@mloskot Is that cmake 2.8 a hard requirement? I would give it a try modernizing it, but it really should be based on a more recent version then.

Well, CMakeLists.txt#L13 SOCI 4.0.0 says

cmake_minimum_required(VERSION 2.8 FATAL_ERROR)

Yes, definitely! In my branch I set the requirement at 3.5 but even that I did consider as an old one.

Your best bet is to discuss the details with current SOCI developers or just pick the version and submit PR, then you will receive feedback.

Is your problem with the installed soci target or building soci itself? I’ll assume the former. For that, you’d likely need to add a find_dependency() call to your sociConfig.cmake. The soci target should already be linking to a sqlite3 target of some kind if it depends on it and in an ideal world, the sqlite3 target would bring with it the necessary header search paths as a transitive dependency. If it doesn’t, then you could explicitly add the missing details to the imported target for soci in your sociConfig.cmake file. The find_dependency() call would be used to locate sqlite3 and then you use the results of that to add the missing info to the soci target.

My CppCon talk from last year may help with that:

@craig.scott Thanks a lot that worked great!

I did work on a first version, which works already quite good. At least it does for sqlite3, using it now is really straight forward.

find_package(SOCI COMPONENTS sqlite3 REQUIRED)

add_executable(Test)
target_link_libraries(Test SOCI::soci_core SOCI::soci_sqlite3)

I haven’t tried it yet, but hopefully using it with FetchContent is as easy as that.

I have my current work here in the feature/modernCmake branch

Reading / Listening about cmake is one thing, using it another. I would really appreciate some feedback on my cmake code, if anyone finds the time! I’m willing to get better at writing cmake code.

@craig.scott I followed exactly your recommendations, but windows dll are not so easy to handle as it should!

see https://github.com/ClausKlein/ModernCppStarter/pull/2/commits/fe76a4416cc053ee19c81e337069fad1c9d52ca3

Nice talk. One more note though: you cannot use $Origin in RPATH for setuid-root applications.

That looks like you’ve tried to export an enum class. I don’t know if that is needed or even valid. Are you saying that is causing a problem, or that you needed to do that to make the project work?

Thanks for reply. I found no way to build the dll for this simple project on windows:

Run cmake --build build --config Debug -j4
7
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
8
Copyright (C) Microsoft Corporation. All rights reserved.
9

10
  Checking File Globs
11
  Checking Build System
12
  Building Custom Rule D:/a/ModernCppStarter/ModernCppStarter/CMakeLists.txt
13
  greeter.cpp
14
  Building Custom Rule D:/a/ModernCppStarter/ModernCppStarter/cpm_modules/fmt/9dd47b889fbf7c876d24be56ca097a884004b789/CMakeLists.txt
15
D:\a\ModernCppStarter\ModernCppStarter\include\greeter/greeter.h(17,21): error C2220: the following warning is treated as an error [D:\a\ModernCppStarter\ModernCppStarter\build\_deps\greeter-build\Greeter.vcxproj]
16
D:\a\ModernCppStarter\ModernCppStarter\include\greeter/greeter.h(17,21): warning C4251: 'greeter::Greeter::name': class 'std::basic_string<char,std::char_traits<char>,std::allocator<char>>' needs to have dll-interface to be used by clients of class 'greeter::Greeter' [D:\a\ModernCppStarter\ModernCppStarter\build\_deps\greeter-build\Greeter.vcxproj]
17
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\include\xstring(4648): message : see declaration of 'std::basic_string<char,std::char_traits<char>,std::allocator<char>>' [D:\a\ModernCppStarter\ModernCppStarter\build\_deps\greeter-build\Greeter.vcxproj]
18
  format.cc
19
  os.cc
20
  Generating Code...
21
     Creating library D:/a/ModernCppStarter/ModernCppStarter/build/_deps/fmt-build/Debug/fmtd.lib and object D:/a/ModernCppStarter/ModernCppStarter/build/_deps/fmt-build/Debug/fmtd.exp
22
  fmt.vcxproj -> D:\a\ModernCppStarter\ModernCppStarter\build\bin\Debug\fmtd.dll
23
Error: Process completed with exit code 1.

You need to disable to treat all warnings as errors or disable warning C4251 when using /MTD or not use the debug runtime

I tried it, see history, -> than it builds, but core dump while test …

IMHO: Warnings are Errors and should never ignored.

Then see the Microsoft page about C4251. It relates to using template classes in an exported class.

For code related warnings, I agree. But generally, no. Neither for MSVC nor GCC. They sometimes add warnings for possible runtime situations that simply do not apply in a specific case.

I found a simple solution for me that helps:
1.) all dll and executables are build in same bin directory
2.) not the class is exported, only the public member functions

set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)