# file GET\_RUNTIME\_DEPENDENCIES picks up incorrect shared objects
**URL:** https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314
**Category:** Code
**Tags:** os:linux
**Created:** [June 3, 2020, 3:19pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314 "2020-06-03T15:19:35Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Stephan268](https://discourse.cmake.org/u/Stephan268)
#### Post date: [June 3, 2020, 3:19pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/1 "2020-06-03T15:19:36Z")
file (GET\_RUNTIME\_DEPENDENCIES ) returns incorrect dependencies to /usr/lib instead of /usr/lib64 on Linux.
Examples are libstdc++.so, libX11.so
The cmake documentation states that libraries are searched on paths returned by calling ldconfig. A call on my computer:
```
/sbin/ldconfig -v -N -X 2>/dev/null | egrep "^([^\t:]*):"
/usr/local/lib64:
/usr/local/lib:
/usr/lib64/graphviz:
/usr/lib64/R/lib:
/lib:
/lib64:
/usr/lib:
/usr/lib64:
```
The system has been “normally” installed and administered.
Maybe the implementation in cmake is overly complex and can be done a lot easier and less error prone. The tool “ldd” returns the full path to the dependent shared objects. These can be taken as is and do not need any search paths, e.g. returned from ldconfig. This works for 64Bit builds as well as 32Bit builds which is probably not the case using the paths returned by ldconfig.
I am running cmake 3.17.2 on Linux.
---
### Author:  [@kyle.edwards](https://discourse.cmake.org/u/kyle.edwards)
#### Post date: [June 3, 2020, 3:22pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/2 "2020-06-03T15:22:52Z")
> [@Stephan268](#):
>
> Maybe the implementation in cmake is overly complex and can be done a lot easier and less error prone. The tool “ldd” returns the full path to the dependent shared objects.
The problem with using `ldd` is that it returns things that might otherwise have been excluded by the `{PRE,POST}_{INCLUDE,EXCLUDE}_REGEX` arguments of `file(GET_RUNTIME_DEPENDENCIES)`.
Cc: @ben.boeckel
---
### Author:  [@Stephan268](https://discourse.cmake.org/u/Stephan268)
#### Post date: [June 3, 2020, 3:28pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/3 "2020-06-03T15:28:55Z")
> [@kyle.edwards](#):
>
> The problem with using `ldd` is that it returns things that might otherwise have been excluded by the `{PRE,POST}_{INCLUDE,EXCLUDE}_REGEX` arguments of `file(GET_RUNTIME_DEPENDENCIES)`
I would have thought that it is up to the user to specify the desired REGEX to match his/her intention.
The current implementation however returns incorrect dependencies. Maybe there is another way to use GET\_RUNTIME\_DEPENDENCIES that I am not aware of.
---
### Author:  [@kyle.edwards](https://discourse.cmake.org/u/kyle.edwards)
#### Post date: [June 3, 2020, 3:41pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/4 "2020-06-03T15:41:26Z")
> [@Stephan268](#):
>
> I would have thought that it is up to the user to specify the desired REGEX to match his/her intention.
It is. The problem is that there is not a way to make `ldd` work with the `REGEX`es, because `ldd` will continue to follow dependencies that would have otherwise been excluded by the `REGEX`es.
For example, if your program depends on `/usr/lib/libfoo.so`, and that in turn depends on `/lib/bar.so`, and you have regexes that exclude `/usr/lib`, then you could filter `/usr/lib/libfoo.so` from the `ldd` output, but you would still get `/lib/bar.so`. With our approach, all of `/usr/lib/libfoo.so`’s dependencies are also excluded (unless they’re pulled in by something else) because it doesn’t make sense to pull in a dependency for something that we’re not including in the first place.
---
### Author:  [@Stephan268](https://discourse.cmake.org/u/Stephan268)
#### Post date: [June 3, 2020, 4:27pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/5 "2020-06-03T16:27:30Z")
> [@kyle.edwards](#):
>
> The problem is that there is not a way to make `ldd` work with the `REGEX` es, because `ldd` will continue to follow dependencies that would have otherwise been excluded by the `REGEX` es.
Thanks for the great example. I was not aware of the fact that ldd returns direct and transitive dependencies. Maybe it would be an idea to use
```
readelf -d myprogram | grep NEEDED
```
to get only the direct dependencies of _myprogram_ and then use
```
ldd myprogram
```
to get the location of the direct dependencies. These could then be used in the regex filters.
---
### Author:  [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 3, 2020, 7:01pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/6 "2020-06-03T19:01:23Z")
What distro are you using? Why are `/usr/lib` and `/usr/lib64` in the list? Seems like a distro configuration problem to me…
How does the loader exclude the libraries it finds in `/usr/lib` first? Does it do an ELF machine flag check?
---
### Author:  [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 3, 2020, 7:05pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/7 "2020-06-03T19:05:05Z")
> [@Stephan268](#):
>
> ```auto
> readelf -d myprogram | grep NEEDED
>
> ```
This is effectively what we do (we also extract RPATH and RUNPATH at this point). However, we then perform the search manually because we _don’t_ want to have `LD_LIBRARY_PATH` interfere here (and we can’t necessarily clear it because it may be necessary to make `ldd` run).
---
### Author:  [@Stephan268](https://discourse.cmake.org/u/Stephan268)
#### Post date: [June 3, 2020, 7:13pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/8 "2020-06-03T19:13:50Z")
> [@ben.boeckel](#):
>
> What distro are you using? Why are `/usr/lib` and `/usr/lib64` in the list? Seems like a distro configuration problem to me…
>
> How does the loader exclude the libraries it finds in `/usr/lib` first? Does it do an ELF machine flag check?
We are using OpenSuse Leap. I am not a Linux specialist and regrettfully cannot tell you why /usr/lib and /usr/lib64 are in the list, what ldconfig is used for and how the loader works. Most probably it needs to have some idea of the architectur else it could not distinguish 32Bit and 64Bit libraries which may reside with the same name in /lib and /lib64.
---
### Author:  [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 3, 2020, 7:16pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/9 "2020-06-03T19:16:44Z")
@kyle.edwards Seems we have to filter libraries based on their machine.
```auto
% readelf -h -d /path/to/bin
OS/ABI: UNIX - GNU
ABI Version: 0
Type: DYN (Shared object file)
Machine: Advanced Micro Devices X86-64
```
---
### Author:  [@Stephan268](https://discourse.cmake.org/u/Stephan268)
#### Post date: [June 3, 2020, 7:49pm UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/10 "2020-06-03T19:49:01Z")
> [@ben.boeckel](#):
>
> However, we then perform the search manually because we _don’t_ want to have `LD_LIBRARY_PATH` interfere here (and we can’t necessarily clear it because it may be necessary to make `ldd` run).
Arghhhh: the usual hassle when using global variables such as environment variables.
My understanding of LD\_LIBRARY\_PATH is that the loader evaluates the variable and gives libraries on LD\_LIBRARY\_PATH precedence over system defaults or other locations where libraries can be found. This means that other libraries may be used at load time than were used at link time. I do not think that this is what is desired when using cmake to detect dependencies for packaging. We want to have exactly those libraries defined when linking. To my understanding this means that LD\_LIBRARY\_PATH may not be set when using ldd for the purpose. If ldd does not work correctly without LD\_LIBRARY\_PATH then there may be a more serious problem on the build system. The approach currently taken by cmake seems to require reimplemenation of some of the logic of the linker/loader which could be very fragile.
Please take all of this with a grain of salt: I have no intimate knowledge of loader/linker and an extreme allergy against global variables.
An article I just found: [LD\_LIBRARY\_PATH – or: How to get yourself into trouble!](https://www.hpc.dtu.dk/?page_id=1180)
---
### Author:  [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 4, 2020, 1:06am UTC](https://discourse.cmake.org/t/file-get-runtime-dependencies-picks-up-incorrect-shared-objects/1314/11 "2020-06-04T01:06:15Z")
> [@Stephan268](#):
>
> If ldd does not work correctly without LD\_LIBRARY\_PATH then there may be a more serious problem on the build system
Cross compilation (though a naked `ldconfig` is a problem in this case anyways I guess), custom binutils… I can’t seem to find the emails where we discussed using `ldd` vs. a re-implementation right now; maybe it was a face-to-face meeting. I seem to remember symlinks being involved now that I’m writing this (`ldd` does `realpath` on its found libraries maybe?). But we do need to search on Windows (because its lookup rules are just searches anyways) and macOS (we also memoize library IDs and whatnot for future bundling).
> [@Stephan268](#):
>
> The approach currently taken by cmake seems to require reimplemenation of some of the logic of the linker/loader which could be very fragile.
It basically is a reimplementation. We had problems with `O(n²)` on big projects when we couldn’t cache the lookups (not all builds happen on SSDs ☹ ). Memoizing found library dependency results cut our times on some projects down by 90% or so (IIRC; they were large in any case). Now some of that was because of the CMake → Python implementation that had been done for that. I don’t know the performance difference between those Python scripts and the resolvers in CMake.