EDITED:
This thread has become more about issues/bugs with file(GET_RUNTIME_DEPENDENCIES) which is completely different than the intended topic. To not confuse future readers I’ve edited the issue.
EDITED:
This thread has become more about issues/bugs with file(GET_RUNTIME_DEPENDENCIES) which is completely different than the intended topic. To not confuse future readers I’ve edited the issue.
@kyle.edwards Do we have examples of file(GRD)
usage? There is the test suite, but that is probably more on the contrived side of things I imagine. Is your common-superbuild migration branch available anywhere (even as a rough draft)?
GET_RUNTIME_DEPENDENCIES has quirks and issues in cmake 3.17.2 so an answer to your question will depend on the build platform.
Short story:
On Linux it does not work as expected so I needed to implement something naiv. The following code is a excerpt from my code and maybe lacking variable declarations, functions definitions, … and elegance but the gist is hopefully clear.
# GET_RUNTIME_DEPENDENCIES does not work on Linux using cmake 3.17.2
# it does not honour the paths given by ldd and searches in system paths itself.
# in doing so it picks up 32 Bit system libs for 64 Bit builds
# this is a naive implementation that uses ldd
if (DEFINED ENV{LD_LIBRARY_PATH})
message (WARNING "LD_LIBRARY_PATH is defined. This may lead to the installation of library versions that were not used at link time.")
endif()
foreach (lib ${all_libs})
execute_process(COMMAND ldd ${lib} OUTPUT_VARIABLE ldd_out)
string (REPLACE "\n" ";" ldd_out_lines ${ldd_out})
foreach (line ${ldd_out_lines})
string (REGEX REPLACE "^.* => | \(.*\)" "" pruned ${line})
string (STRIP ${pruned} dep_filename)
if (IS_ABSOLUTE ${dep_filename})
is_system_lib (${dep_filename} sys_lib)
if (sys_lib EQUAL 0 OR INSTALL_SYSLIBS STREQUAL "true")
list (FIND dependencies ${dep_filename} found)
if (found LESS 0)
list (APPEND dependencies ${dep_filename})
endif()
endif()
endif()
endforeach()
endforeach()
On windows it works but has problems with API_SETS. Small code snippet
if (CMAKE_HOST_SYSTEM_NAME STREQUAL "Windows")
#[[
The following statements are a workaround of problems that cmake 3.17 has with Windows API sets.
Reference: https://ofekshilon.com/2016/03/27/on-api-ms-win-xxxxx-dll-and-other-dependency-walker-glitches/
#]]
LIST(APPEND pre_exclude_regexes "api-ms-.*") # windows API
LIST(APPEND pre_exclude_regexes "ext-ms-.*") # windows API
LIST(APPEND pre_exclude_regexes "ieshims.dll") # windows API
LIST(APPEND pre_exclude_regexes "emclient.dll") # windows API
LIST(APPEND pre_exclude_regexes "devicelockhelpers.dll") # windows API
LIST(APPEND post_exclude_regexes ".*WINDOWS[\\/]system32.*") # windows system dlls
file(GET_RUNTIME_DEPENDENCIES
RESOLVED_DEPENDENCIES_VAR dependencies
LIBRARIES ${all_libs}
DIRECTORIES ${TBT_PATH_EXTERNAL}
CONFLICTING_DEPENDENCIES_PREFIX conflicts
PRE_EXCLUDE_REGEXES ${pre_exclude_regexes}
POST_EXCLUDE_REGEXES ${post_exclude_regexes}
POST_INCLUDE_REGEXES ${post_include_regexes}
)
list (LENGTH conflicts_FILENAMES num_files_conflict)
if (${num_files_conflict} GREATER 0)
message(WARNING "dependencies of ${lib} found at multiple locations will not be installed.")
foreach (lib_conflict ${conflicts_FILENAMES})
message(STATUS "${lib_conflict} found at: ${conflicts_${lib_conflict}}")
endforeach()
endif()
endif()
Hmm. This is news to us. AFAIK, it is working as expected, though multilib or multiarch installations could certainly confuse it. Note that cross-compilation is not supported at the moment (see this issue). Could you please file an issue about what is going on? @kyle.edwards
See this question for how I think CMake should provide these pre-exclude regexes.
I suspect that’s mostly an issue of rpaths (or lack thereof), and the fact that shared libraries can’t just be copied around willy-nilly on Linux, no?
For example, /usr/bin/bash
as installed on my Fedora 33 system doesn’t really have a runtime dependency on the file /lib64/libc.so.6
at all — it has a dependency on the name libc.so.6
. Only the system’s dynamic loader is equipped to resolve that name usefully. So just knowing the path to /lib64/libc.so.6
doesn’t really do you a whole lot of good all by itself.
I am not a Linux specialist, so please take what I am writing as my impression and not my definite view.
Well, that’s certainly possible. I’m more curious whether it is our code which breaks the “works as expected” or there’s a misunderstanding of how ELF library searching and loading works.
I believe you mean runtime linker, but the thing is that ldd
gives the full recursive list of libraries found. That interferes with the cross-platform logic we want to use for the pre- and post- include and exclude regexes. We can certainly inspect the binaries we find and do architecture matching to exclude libraries which are not viable (though there are various reasons a library might be unsuitable, architecture matching is probably fine for 90% of the use cases). (Like I said, cross-compilation is definitely not yet supported and multi-lib and multi-arch setups could use some more testing.)
What I think you want to do is to use $ORIGIN
as the (basis of) your install time rpath. You’ll want to edit libraries you copy into your package as well (this kind of stuff is planned for a future API by CMake, but it is much more complicated because the platform differences are far greater in this region). file(GRD)
is based on the strategy we’ve been using for creating ParaView packages for the past few years which has been working really well.
It’s a bit more complex than that, even — the runtime loader keeps a cache (maintained by ldconfig
) which maps between library symbols and the .so
files that provide them. The cache is normally populated by scanning all of the shared library files in any of the directories configured in /etc/ld.so.conf
(or, if utilized by the Linux distro, by the drop-in files added to /etc/ld.so.conf.d/
), in addition to the predefined standard paths.
But you’re right that in the build tree, libraries not linked through standard -llibname
means (such as most IMPORTED
targets from other CMake projects’ Config.cmake
files) will be referenced by their full disk path. Dynamic linking cares not for such things, though (only the library SONAME will be stored in the binary’s symbol table, even if the compiler is given a full path), so if the file isn’t in a standard system dir, its location will also be set as an RPATH
on the build-tree binary so that the library is preferentially loaded from there.
In its default install mode, CMake removes all RPATHs, which means that a binary’s shared library dependencies can and will change from the build-time linking if there are different versions of the same shared library installed on the system, compared to what it was linked with when it was built.
There are also several target properties that affect all of that, some in non-obvious / confusing ways.
BUILD_WITH_INSTALL_RPATH
will avoid the build-tree-only RPATHs
that make compiled libraries and executables runnable from the build tree even if they reference uninstalled libs. Which mostly just serves to make your binary unable to be run from the build tree unless it’s only linked with installed libraries. (OTOH, if using an INSTALL_RPATH
that can be a good way of verifying that it’s set correctly.)BUILD_RPATH_USE_ORIGIN
will cause any linking within the build tree to use relative paths. Since it doesn’t affect any linking outside the build tree. It’s really only useful for making your build tree relocatable, on the odd chance you need to do that.INSTALL_RPATH_USE_LINK_PATH
adds non-standard dependency locations to the installed RPATH
of any binary it’s set on, instead of the default blanking. Which can be extremely helpful, though it has one major shortcoming: It only works on outside dependencies. If your project builds, say, a shared library and an executable linked with it, and you install that project in a non-standard location, your executable’s still not going to be able to find your own shared library unless you set an appropriate INSTALL_RPATH
. (Or if something like an INSTALL_RPATH_USE_ORIGIN
is added to CMake.)Yeah, it will affect the results if set, same as any tool that uses the system loader — including ldd
. On Linux GET_RUNTIME_DEPENDENCIES
is just a wrapper for calls to objdump -p
. (By default, anyway; that can be changed by setting CMAKE_GET_RUNTIME_DEPENDENCIES_COMMAND
.) In this context objdump
only really differs from ldd
in that it’s not recursive — CMake does its own recursion, though. The output of objdump -p
is then parsed for RPATH
, RUNPATH
, and NEEDED
lines, to collect dependency details.
Ouch. Yeah, when I said the “only difference” between objdump
and ldd
was recursion, I was wrong. objdump
also has no native awareness of architecture, the tool has to account for that itself. GET_RUNTIME_DEPENDENCIES
fails to, it seems, so I’m seeing the same thing here.
In fact, on my system (CMake 3.18.4 on Fedora 33 x86_64), it seems GET_RUNTIME_DEPENEDENCIES
will prefer the 32-bit version of any multiarch library installed on the system, maybe just because the path is shorter.
So, e.g. an installed /lib/libQt5core.so.5
gets returned for a 64-bit Qt executable that’s really linked with /lib64/libQt5core.so.5
. And that’s true even if it also returns /lib64/libQt5Gui.so.5
because /lib/libQt5Gui.so.5
is not installed. So it not only picks the wrong arch, but has no protections against mixing 32-bit and 64-bit libraries. That’s… kind of a mess.
I’ll also add that GET_RUNTIME_DEPENDENCIES
is DOG slow, holy crap. I think it’s actually executing ldconfig
and re-parsing its entire output for each individual dependency encountered. Which, on a desktop system where ldconfig -N -p -X |wc -l
shows 5428 lines, is unbearably glacial.
For my executable with these dependencies:
Dynamic Section:
NEEDED libpthread.so.0
NEEDED libMagick++-6.Q16.so.8
NEEDED libMagickWand-6.Q16.so.6
NEEDED libMagickCore-6.Q16.so.6
NEEDED libjsoncpp.so.24
NEEDED libQt5Widgets.so.5
NEEDED libQt5Gui.so.5
NEEDED libQt5Core.so.5
NEEDED libavcodec.so.58
NEEDED libavdevice.so.58
NEEDED libavformat.so.58
NEEDED libavfilter.so.7
NEEDED libavutil.so.56
NEEDED libpostproc.so.55
NEEDED libswscale.so.5
NEEDED libswresample.so.3
NEEDED libavresample.so.4
NEEDED libgomp.so.1
NEEDED libzmq.so.5
NEEDED libstdc++.so.6
NEEDED libm.so.6
NEEDED libgcc_s.so.1
NEEDED libc.so.6
a ONE MINUTE FORTY-FIVE SECOND pause occurs during cmake --install
, due to this code which does nothing but display the results:
install(CODE [[
file(GET_RUNTIME_DEPENDENCIES
EXECUTABLES $<TARGET_FILE:myexe>
RESOLVED_DEPENDENCIES_VAR _r_deps
UNRESOLVED_DEPENDENCIES_VAR _u_deps
)
foreach(_file ${_r_deps})
message(STATUS "Dependency: ${_file}")
endforeach()
]])
…and after all that wait, then shows a whole bunch of incorrect 32-bit library paths mixed in with the 64-bit ones.
FTR: The arch issue comes about because the output of ldconfig
will list all libs installed in any architecture, and at least on Fedora systems they all have the same names, just at different paths. So, e.g. this is how they’re listed in the cache:
$ ldconfig -N -p -X |grep Qt5Core
libQt5Core.so.5 (libc6,x86-64, OS ABI: Linux 3.17.0) => /lib64/libQt5Core.so.5
libQt5Core.so.5 (libc6, OS ABI: Linux 3.17.0) => /lib/libQt5Core.so.5
libQt5Core.so (libc6,x86-64, OS ABI: Linux 3.17.0) => /lib64/libQt5Core.so
Path length doesn’t matter; path order does.
As for speed…I haven’t noticed it with the Python implementation so much, but I also haven’t ported over yet (time…) and that does do forking for objdump
for each library. There’s only one ldconfig
call to get paths though (-v -N -X
) and the grep you need is for '^(/.*):'
(basically, the real one is a bit more specific). On my machine, that’s 3 paths, but I also don’t have any 32bit libraries installed either.
I really meant the “build time linker” in order to make sure I package exactly what I explicitly or implicitly linked against into the kit. This is done to make the kit less dependent on what is installed on the target system, e.g. X11.
I am pretty sure this is not the case on my system because it actually picked up libraries in /lib and /usr/lib for a 64 Bit build. ldd does not.
Yes: I change rpath to $ORIGIN for all “system libraries” during cmake install step. I use patchelf for this purpose because I did not find out how to do this using cmake.
I also noticed this and whilst not a reason for my naiv substitute for GET_RUNTIME_DEPENDENCIES was a pleasant side effect.
But all we have is the runtime linker information; the build link line is long gone by the time we get the final library or executable. If you build with mixed up RPATH information, we’re going to be just as confused as the runtime linker.
I said “can certainly inspect” in that we could, but do not currently do so.
Yep. The idea is that we’ll eventually have CMake APIs which will patch up libraries as needed using the system tools, but designing that API is difficult due to the vast differences between what platforms need.
Probably my implementation coincedentally works in combination of the way the .so are built and the use of ldd:
This is essentially what we’re doing (minus the actual install and patching), but we don’t let ldd
do the recursion so that we can have consistent behavior on every platform.
But then cmake should never pick up .so from /lib for a 64Bit build.
Idea: Use ldd on the top level to compute all dependencies (incl. transitive) with their full path and then use objdump to figure out the direct dependencies. Then it would be possible to filter using the PRE and POST filters of GET_RUNTIME_DEPENDENCIES.
Well, we will once we start rejecting due to architecture mismatches in ELF binaries.
Great! Then it remains to be decided if considering LD_LIBRARY_PATH and output of ldconfig is good. My gut feeling is that they are part of the “runtime” enviroment and should not be condidered in packaging. Please keep in mind that I am not a Linux wizard, so this only based on “feeling” and not on knowledge.
We have to use this since this is the only reliable way of determining what directories should be searched at all. Different distros use different layouts (/usr/lib
(32bit on Red Hat, native on Debian pre-multiarch), /usr/lib64
(64bit on Red Hat), /usr/lib/$triple
(Debian multiarch), GoboLinux layouts, etc.). I don’t think we use LD_LIBRARY_PATH
(it is only mentioned in documentation snippets about ELF loading that we have).