GET_RUNTIME_DEPENDENCIES has difficulties with windows API sets

Code
#1

GET_RUNTIME_DEPENDENCIES return dlls that do not physically exist and are part of the concept that Microsoft refers to as API sets.

Examples:

  • api-ms-win-core-libraryloader-l1-2-1.dll
  • api-ms-win-core-atoms-l1-1-0.dll
  • api-ms-win-core-winrt-error-l1-1-1.dll
  • api-ms-win-core-sidebyside-l1-1-0.dll
  • api-ms-win-core-localization-obsolete-l1-3-0.dll
  • api-ms-win-core-heap-l1-2-0.dll
  • api-ms-win-core-heap-l2-1-0.dll
  • api-ms-win-core-delayload-l1-1-1.dll
  • api-ms-win-core-libraryloader-l1-2-0.dll
  • api-ms-win-core-rtlsupport-l1-2-0.dll
  • api-ms-win-core-shlwapi-obsolete-l1-2-0.dll
  • api-ms-win-security-base-l1-2-0.dll
    There are other with the prefix “ext-ms-win” or “ext-ms”.

I can add exclusion patterns to GET_RUNTIME_DEPENDENCIES using pre_exclude_regexes but this is very fragile, because it depends on OS versions. I think it would be better if cmake handles these tricky aspects. It would fit better into the philosophy of cmake to provide abstraction of the OS, makesystem, …

References: On API-MS-WIN-XXXXX.DLL, and Other Dependency Walker Glitches

How to disable system directory search for file(GET_RUNTIME_DEPENDENCIES
get_prerequisites() tries to resolve system libraries even though advised not to
How do I pull full paths out of UNRESOLVED_DEPENDENCIES_VAR
#2

And why did MS added those to the VC runtime installers for e.g. Windows XP? Because sometimes they are needed.

#3

I probably did not phrase the problem I have with the implementation in cmake very well.

I use GET_RUNTIME_DEPENDENCIES to figure out which external DLLs I need to add to my application kit. This can be tricky because I need to classify into “own”, “used 3rd party”, “part of the OS”, “part of the VC runtime”.
Ideally cmake could help classify the latter two classes.
I would not add “part of the OS” to my kit.
I would not add “part of the VC runtime” to my kit but reference it so that it can be installed seperatly.

The API set “DLLs” are (to my knowledge) not a physical DLL but an abstract concept which is part of the OS. So actually they should never be output by GET_RUNTIME_DEPENDENCIES.

#4

They are “part” of Windows 8 and later. This is a problem when deploying to older Windows versions. That’s why they are part of the redistributable files.

#5

Aha! Maybe the problem I am facing has a different reason. I am running Visual Studio 2015 on Windows 10.

file (GET_RUNTIME_DEPENDENCIES …)

returns

file Could not resolve file api-ms-win-crt-convert-l1-1-0.dll

Searching for api-ms-win-crt-convert-l1-1-0.dll locates in a multitude of locations, e.g. Appdata or installation directory of MS-Teams, MS-Office, MS-Onedrive, firefox, …
as well as
C:\Programs(x86)\Windows Kits\10\Redist…
C:\Programs(x86)\Microsoft Visual Studio 14.0\Common7…

As I build with VS2015 I would have expected that cmake locates the DLL in the installation directory of VS2015. Maybe it would even be better if it additionally treats it as something special, i.e. part of a redistributable. This would allow the programmer to decide if he/she wants to add the DLLs to his/her own kit or have it installed via the original Microsoft redistributable package.

With the way it is now I do not know if I should treat unresolved files as error. I could maintain a whitelist. This would force me to update this list with every new VS or redistributable version. This might better be done with cmake. Also I cannot add these files to my install kit because my code does not know where they are located.

Ideas warmely welcome.

#6

I did this in the Python prototype the CMake implementation was based on. I remember @kyle.edwards and I discussing these cases, but can’t remember exactly what we decided to do about them. It seems that yes, PRE_EXCLUDE_REGEXES is the thing to do right now. Maybe we could add something like PRE_EXCLUDE_SYSTEM_LIBRARIES which is defined to be some sensible, but still restricted set of libraries for each platform (libSystem.dylib, api-ms-*.dll (and friends), and libc.so is about as high as I think we’d want to go).

#7

Though this does open up a huge hole of keeping compatibility if we ever try to change it… Maybe we just have regexes for classes of libraries in a module?

# CMakeSystemLibraryRegexes.cmake
set(CMAKE_SYSTEM_WINDOWS_API_DLLS "...")
set(CMAKE_SYSTEM_APPLE_LIBSYSTEM "...")

Then users can use them as wanted in PRE_EXCLUDE_REGEXES and we can add new variables as new “classes” of libraries appear?

file(GET_RUNTIME_DEPENDENCIES) issues
#8

At first glance the provisioning of regex looks good. It could give users a good starting point for further modifications. Some ideas and design considerations.

  • the regex should probably depend on compiler+version, build platform, target platform
  • the required classes will need some thought especially if they abstract well from the OS
#9

I think it’s sufficient to have a blanket exclusion for the pseudo DLL names that never actually is exist. MSVC runtime DLLs might warrant a per-toolchain regex, but who is packaging up more than one at a time anyways?

#10

It seems that my initial assumption that windows API sets are the problem is incorrect. Hendrik Sattler’s reply (see above) started me having a closer look at the topic. See replies below.

I now have an additional dll that cannot be resolved: “gdiplus.dll”. It seems that GET_RUNTIME_DEPENDENCIES does not search in all required directories. The problem occurs on a Windows 7 build machine.

#11

Is therre any solution to this exceept the exclusion regexes?

#12

Not today. Eventually, it’d be nice to have a module that provides the regexes for easy use (MRs welcome :slight_smile: ), but there’s no other mechanism that is necessary.

#13

Thanks, for your reply, is there alreardy a Bug rerport ffor this or is the implementation forr the logic of the runtime loading mechanism corrrect in GET_RUNTIME_DEPENDENCIES under Windows.

I am just currious as cerrtain librraries such as emclient.dll pop up which have nothing to do with the traversed Dependency Graph

#14

Libraries that show up are listed somewhere in your dependency chain in some way. If you can show an example which reports a DLL that is not actually used, that would go a long way to figuring out what’s wrong.

#15

The strange dependencies come in by the following:

Delay Load Dependencies from Dumbin.exe with coreuicomponents.dll (system lib)
Maybe the delay load can be ignored, or made optionally ignored, but probaly not always the case…

image
image759Ă—532 101 KB

Also urlmon.dll has some strange delay load dependencies wpaxholder.dll

#16

Hmm. I thought delayload dependencies were ignored. @kyle.edwards?

#17

On Windows, GET_RUNTIME_DEPENDENCIES simply scans the output of dumpbin (or objdump) for .dll. It doesn’t distinguish between delay load and normal dependencies.

#18

Ah, the prototype extractor from the superbuild bails when it sees “delay load dependencies”. I suppose a flag to handle this is likely warranted (possibly with a policy to change the default).

1 Like
#19

That would be great. Or might there be an option to set arguments to the dumbbin.exe?. Actually the depedency tool can be set (but no arguments), but CMake then does the parsing, not sure how that should work to set another tool. I guess only the supported ones on the doc pages are meaningful.
Otherwise maye support a tool which outputs JSON in some sensful format, which can then be parsed platform independent etc…

#20

In the long term, I think the goal should be for CMake to parse the PE file format directly and not use external tools like dumpbin and objdump. The format is well-documented.