Avoid GuessLibrarySOName on Windows

Development
1

Hi,

We’re adding external libraries using full paths in target_link_libraries. On Windows, this points to a .lib file. During link information processing, .lib files are treated as shared libraries, which triggers a call to cmOrderDirectories::AddRuntimeLibrary, and subsequently to cmOrderDirectoriesConstraintSOName.

This leads to GuessLibrarySOName instantiating cmElf, which opens the file to check if it’s an ELF library and then (obviously) always return false. However, file access on Windows is slow — and with corporate security tools, even slower. In one of our projects, the CMake generation phase takes around 150 seconds, and roughly 30% of that time (about 50 seconds) is spent on these unnecessary file reads. By short-circuiting GuessLibrarySOName to return false early, we can avoid opening and reading the file, significantly improving performance.

Our project includes many targets, external libraries, and four build configurations. We’ve tested this behavior with CMake 3.30 and 4.x — both exhibit the same issue. We use the “Visual Studio 17 2022” generator.

Question:
Is there a recommended way to avoid these unnecessary file reads on Windows?
It seems that both .lib and .dll are treated as shared libraries, which is why the ELF check is triggered.

2

Create imported libraries so CMake knows their type without guessing.

add_library(foo STATIC IMPORTED)
set_property(TARGET foo PROPERTY IMPORTED_LOCATION "c:/path/to/foo.lib")
#...
target_link_libraries(myThing PRIVATE foo)
3

I experimented with using an interface-imported library alongside multiple statically imported libraries. On Linux, this setup introduces complexity due to the possibility of (true) shared libraries (.so files). To optimize, I switched from specifying full paths to using the AddImportedLibraryfunction below for some external packages that contain many library files. This change initially reduced generation time. However, when I applied it to more packages, the time increased again.

I didn’t investigate this further, assuming there must be a simpler way to avoid opening files on Windows just to determine whether they are ELF binaries. One thing I still find confusing is why .lib files on Windows are interpreted as shared libraries. Isn’t a .lib always linked the same way, regardless of whether it’s a static or import library?

Here’s the function I used:


function(AddImportedLibrary)
  set(options SHARED)
  set(oneValueArgs NAME)
  set(multiValueArgs DEBUG RELEASE INCLUDES)

  cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})

  if(NOT ARGS_DEBUG)
    set(ARGS_DEBUG ${ARGS_RELEASE})
  else()
    list(LENGTH ARGS_DEBUG NBRDEB)
    list(LENGTH ARGS_RELEASE NBRREL)
    if(NOT (NBRDEB EQUAL NBRREL))
      message(FATAL_ERROR "DEBUG and RELEASE do not have the same length.")
    endif()
  endif()

  add_library(${ARGS_NAME} INTERFACE IMPORTED)
  target_include_directories(${ARGS_NAME} INTERFACE ${ARGS_INCLUDES})

  foreach(LIBD LIBR IN ZIP_LISTS ARGS_DEBUG ARGS_RELEASE)
    get_filename_component(FNAME "${LIBR}" NAME_WE)
    set(LIBNAME "${ARGS_NAME}::${FNAME}")
    if(ARGS_SHARED)
      add_library(${LIBNAME} SHARED IMPORTED)
    else()
      add_library(${LIBNAME} STATIC IMPORTED)
    endif()
    set_target_properties(${LIBNAME} PROPERTIES
      IMPORTED_LOCATION ${LIBR}
      IMPORTED_LOCATION_DEBUG ${LIBD})
    target_link_libraries(${ARGS_NAME} INTERFACE ${LIBNAME})
  endforeach()
endfunction()
4

Your AddImportedLibrary function is creating more than one imported target per library, which may explain increased generation time. A single imported target can represent both the library artifact(s) for all configurations and the usage requirements, except that we don’t model the possibility that a library is shared in one configuration and static in another.

why .lib files on Windows are interpreted as shared libraries.

To what code are you referring?

5

The relevant call stack is:

cmSystemTools::GuessLibrarySOName
cmOrderDirectoriesConstraintSOName::cmOrderDirectoriesConstraintSOName
cmOrderDirectories::AddRuntimeLibrary
cmComputeLinkInformation::AddLibraryRuntimeInfo

In AddLibraryRuntimeInfo, the check for whether a file is a shared library is done via:

is_shared_library = this->ExtractSharedLibraryName.find(file);

This condition returns true if the file name ends with .dll or .lib.

Behavior of AddImportedLibrary

The AddImportedLibrary function creates one imported interface target per external package. Each package may contain multiple library files (.lib, .so or .a), and for each of these, a separate target is added.

All these targets are either marked as STATIC or SHARED depending on the SHARED flag passed to the function. This behavior is consistent across configurations (Debug or not Debug) and does not vary per file.

6

This is a case of an abstraction leaking across platforms. ExtractSharedLibraryName gains.lib via CMAKE_IMPORT_LIBRARY_SUFFIX because that is the suffix for linking to a shared library on Windows, via its import library. The is_shared_library code path is probably not needed on Windows, so additional conditions/heuristics could be added to avoid it.

7

Thanks for the clarification — this aligns perfectly with the issue I’ve been encountering.

Do you have any estimate on when a fix might be available? I’d really prefer to retain the simplicity of adding external dependencies via full paths, rather than switching to add_library. Unfortunately, the built-in find_package modules also rely on the less optimal approach of returning full paths in variables, which makes them unsuitable.

We’ve put a lot of effort into optimizing the configure step, but the generation phase on Windows continues to be a significant performance bottleneck. Removing unnecessary file reads is definitely a step in the right direction. That said, the real game changer on Windows would be the ability to parallelize the writing of project files, as file I/O is currently a major time sink.

8

I’ve opened CMake Issue 27214 for this.