find_file() not resolving relative paths correctly (?)

Code
1

Hi,

I’m using find_file to try to locate where builtin compiler library (in this case tsan.so) is located. I’m doing this by first parsing the output of try_compile(... -print-search-dirs) to extract the list of paths, and then calling find_file with that list to locate the given library:

function(try_search_sanitizer_library variable _name)
  execute_process(COMMAND ${CMAKE_C_COMPILER} -print-search-dirs
                  OUTPUT_VARIABLE cc_search_dirs)
  # Extract the line listing the library paths
  string(REGEX MATCH "libraries: =(.*)\n" _ ${cc_search_dirs})
  # CMAKE expects lists to be semicolon-separated instead of colon.
  string(REPLACE ":" ";" cc_library_dirs ${CMAKE_MATCH_1})
  message(STATUS "cc_library_dirs: ${cc_library_dirs}")
  find_file(${variable} ${_name} PATHS ${cc_library_dirs} NO_DEFAULT_PATH)
endfunction()

This is called as try_search_sanitizer_library(TSan_path libtsan.so.0).

However, it’s failing to find the correct libtsan, picking up another one on my machine.

If I enable CMAKE_FIND_DEBUG_MODE, and simplify to hardcode the “correct” path libtsan is in, find_file() seems to be getting the relative path resolution wrong (!):

function(try_search_sanitizer_library variable _name)
   # TEST: Just hardcode the correct path:
set(CMAKE_FIND_DEBUG_MODE TRUE)
  find_file(${variable} ${_name} PATHS "/usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64" NO_DEFAULT_PATH)
set(CMAKE_FIND_DEBUG_MODE FALSE)
endfunction()

This gives the following output:

CMake Debug Log at tlm/cmake/Modules/CouchbaseSanitizers.cmake:14 (find_file):
  find_file called with the following settings:

    VAR: TSan_path
    NAMES: "libtsan.so.0"
    Documentation: Path to a file.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 0
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 0
      Search AppBundle First: 0
    NO_DEFAULT_PATH Enabled

  find_file considered the following locations:

    /usr/lib64/libtsan.so.0

  The item was not found.

Note that CMake appears to have resolved that relative path to /usr/lib64. However, if I actually check with realpath I see:

$ realpath /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64
/usr/local/lib64

And indeed if I list that it exists:

$ ls -l /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64/libtsan.so
lrwxrwxrwx 1 root root 16 Mar  9  2018 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64/libtsan.so -> libtsan.so.0.0.0

So - does find_file() not handle relative paths correctly?

2

From your description, what I see is you try to search for libtsan.so.0 but, by hand, you check that libtsan.so exists! Which is, obviously, different from libtsan.so.0.

3

@marc.chevrier True, I was inconsistent in what I posted. However that file also exists (both are symlinks):

$ ls -l /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64/libtsan.so.0
lrwxrwxrwx 1 root root 16 Mar  9  2018 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.3.0/../../../../lib64/libtsan.so.0 -> libtsan.so.0.0.0

I believe the crux of this issue is how the relative path including symlinks is resolved by CMake - note how realpath resolves to /usr/local/lib64, yet CMake attempts to look in /usr/lib64.

When I get chance (next week) I’ll try to isolate this to a simpler scenario - exactly what relative path / symlink combo seems to be causing the problem.

However - is this a known issue / limitation with find_file()? Is there a workaround?

4

Ok, I’ve reproduced this in a simpler form. The issue looks to be that find_file() resolves relative path components before it resolves symlinks, which is inconsistent with normal Linux path resolution. See the following CMake 3.18.4 output which searches using find_file() with both direct and symlink path; then cat just to prove normal Linux shell works.

I’m unable to upload a tar of the sample, but it’s easy enough to re-create. First the CMakeLists.txt:

cmake_minimum_required(VERSION 3.13)
project(FindPathSymlink LANGUAGES)

set(CMAKE_FIND_DEBUG_MODE TRUE)
find_file(var_direct file.so PATHS "${CMAKE_SOURCE_DIR}/usr/local/lib64" NO_DEFAULT_PATH)
find_file(var_symlink file.so PATHS "${CMAKE_SOURCE_DIR}/usr/7.3.0/../lib64/file.so" NO_DEFAULT_PATH)
execute_process(COMMAND cat ${CMAKE_SOURCE_DIR}/usr/7.3.0/../lib64/file.so
    OUTPUT_VARIABLE var_cat)

message(STATUS "find_file(usr/local/lib64): ${var_direct}")
message(STATUS "find_file(usr/7.3.0/../lib64): ${var_symlink}")
message(STATUS "cat 'usr/7.3.0/../lib64/file.so': ${var_cat}")

Then create the following directory / file layout:

mkdir -p usr/local/lib64
echo "Found you!" > usr/local/lib64/file.so 
mkdir -p usr/local/7.3.0
cd usr/
ln -s local/7.3.0 7.3.0

You should end up with the following layout:

$ find . -ls
  3571714      4 drwxr-xr-x   3 couchbase couchbase     4096 Oct 12 09:11 .
  3571715      4 drwxr-xr-x   3 couchbase couchbase     4096 Oct 12 09:03 ./usr
  3553162      0 lrwxrwxrwx   1 couchbase couchbase       11 Oct 12 09:03 ./usr/7.3.0 -> local/7.3.0
  3571716      4 drwxr-xr-x   4 couchbase couchbase     4096 Oct 12 09:02 ./usr/local
  3571717      4 drwxr-xr-x   2 couchbase couchbase     4096 Oct 12 09:01 ./usr/local/lib64
  3553107      4 -rw-r--r--   1 couchbase couchbase       11 Oct 12 09:09 ./usr/local/lib64/file.so
  3571718      4 drwxr-xr-x   2 couchbase couchbase     4096 Oct 12 09:02 ./usr/local/7.3.0
  3553900      4 -rw-r--r--   1 couchbase couchbase      589 Oct 12 09:09 ./CMakeLists.txt

Running this gives:

CMake Debug Log at CMakeLists.txt:5 (find_file):
  find_file called with the following settings:

    VAR: var_direct
    NAMES: "file.so"
    Documentation: Path to a file.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 0
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 0
      Search AppBundle First: 0
    NO_DEFAULT_PATH Enabled

  find_file considered the following locations:

  The item was found at

    /home/couchbase/find_file_symlinks2/usr/local/lib64/file.so



CMake Debug Log at CMakeLists.txt:6 (find_file):
  find_file called with the following settings:

    VAR: var_symlink
    NAMES: "file.so"
    Documentation: Path to a file.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 0
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 0
      Search AppBundle First: 0
    NO_DEFAULT_PATH Enabled

  find_file considered the following locations:

    /home/couchbase/find_file_symlinks2/usr/lib64/file.so/file.so

  The item was not found.



-- find_file(usr/local/lib64): /home/couchbase/find_file_symlinks2/usr/local/lib64/file.so
-- find_file(usr/7.3.0/../lib64): var_symlink-NOTFOUND
-- cat 'usr/7.3.0/../lib64/file.so': Found you!

-- Configuring done
-- Generating done
-- Build files have been written to: /home/couchbase/find_file_symlinks2/build
5

This sounds like it’d be a policy to fix. Could you please file an issue with this information please?

Cc: @brad.king

6

Done: https://gitlab.kitware.com/cmake/cmake/-/issues/21502

1 Like