IntelLLVM: CMake 3.25+ breaks link.exe options without LINKER prefix

Development
,
1

IntelLLVM: CMake 3.25+ breaks link.exe options without LINKER prefix

In CMake 3.25 (specifically https://gitlab.kitware.com/cmake/cmake/-/merge_requests/7533) IntelLLVM changed to pass user provided linker flags (target_link_options) etc. to the driver instead of the linker. To pass options to the linker users need to use the LINKER: prefix for these options. Traditionally the linker options don’t need escaping in MSVC or toolchains that mimic the MSVC command line like clang-cl.

Simple Reproducer 

cmake_minimum_required(VERSION 3.25)

project(test_link)

add_executable(main main.cpp)

target_link_options(main PUBLIC /LIBPATH:${CMAKE_CURRENT_SOURCE_DIR} lib.lib)



# Create the source file

"int main(){}" | Out-File -FilePath main.cpp

# Create an empty library

New-Item lib.cpp

cl /c lib.cpp

lib lib.obj

# Configure and build with Ninja

cmake -S . -B build -DCMAKE_CXX_COMPILER=icx -G Ninja

cmake --build build -- -v

Results in


[2/2] Linking CXX executable main.exe

FAILED: main.exe

C:\WINDOWS\system32\cmd.exe /C "cd . && (...)\cmake\build\bin\cmake.exe -E vs_link_exe --msvc-ver=1929 --intdir=CMakeFiles\main.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\mt.exe --manifests -- (...)\bin\icx.exe /nologo CMakeFiles\main.dir\main.cpp.obj /Qoption,link,/machine:x64 /Qoption,link,/debug /Qoption,link,/INCREMENTAL /Qoption,link,/subsystem:console /LIBPATH:(...)/cmake/test_icx lib.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /link /out:main.exe /implib:main.lib /pdb:main.pdb /version:0.0 && cd ."

LINK: command "(...)\bin\icx.exe /nologo CMakeFiles\main.dir\main.cpp.obj /Qoption,link,/machine:x64 /Qoption,link,/debug /Qoption,link,/INCREMENTAL /Qoption,link,/subsystem:console /LIBPATH:(...)/cmake/test_icx lib.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /link /out:main.exe /implib:main.lib /pdb:main.pdb /version:0.0 /MANIFEST:EMBED,ID=1" failed (exit code 1104) with the following output:

icx: warning: unknown argument ignored in clang-cl: '-LIBPATH:D:/iusers/gergelym/cmake/test_icx' [-Wunknown-argument]

LINK : fatal error LNK1104: cannot open file 'lib.lib'

icx: error: linker command failed with exit code 1104 (use -v to see invocation)

ninja: build stopped: subcommand failed.

Given that this affects major projects like LLVM and its a big departure from traditional behavior on Windows, following a discussion with @williamr and @Harini, Intel would like to restore compatibility with CMake scripts that rely on the pre 3.25 behavior.

Proposed solution

  • Add a new CMake Policy CMP<XXX> for controlling linker flags behaviour for IntelLLVM.

  • If CMP<XXX> is set to OLD: Windows-IntelLLVM.cmake will instruct CMake to pass the linker flags to the compiler driver, compatible with 3.25 behaviour.

  • If CMP<XXX> is set to NEW: CMake is instructed to use the linker directly. This should be compatible with pre 3.25 (and it is the exact same as pre 3.23).

  • Add new CMake linker type DEFAULT_IPO, used when linking targets that have the INTERPROCEDURAL_OPTIMIZATION property, and LINKER_TYPE property is unset. If the toolchain

does not define CMAKE_{lang}_USING_LINKER_DEFAULT_IPO then DEFAULT_IPO is the same as DEFAULT. This matches the beahviour of the icx compiler driver, and makes IPO work “out of the box” for IntelLLLVM

  • Only when CMP<XXX> is set to NEW Windows-IntellLLVM.config sets CMAKE_{lang}_USING_LINKER_DEFAULT_IPO to use the lld-link.exe bundled with the compiler (found using icx --print-prog-name=lld-link /nologo)

  • (Optional) Extend CMake to allow toolchains to define _CMAKE_${lang}_IPO_SUPPORTED_LINKER_TYPES. Enabling IPO when the LINKER_TYPE property is set to something else raises an error. This gives a clearer and earlier

error for users that change LINKER_TYPE and enable IPO.

Open Questions

  • Should the 3.25+ behaviour be deprecated? If both should be supported then an option instead of a policy may be added. Such an option could come in handy for https://gitlab.kitware.com/cmake/cmake/-/issues/24243 too.

  • Can the policy be “backdated”? If a project is asking for compatibility with cmake version

earlier than 3.25 (or maybe 3.23) can CMP<XXX> default to NEW?

  • Can/Should a simpler version of the fix (revert the 3.25 change based on the value of the policy) be backported to maintained bugfix releases of CMake?

  • Could the policy be per target? This allows mixing the two styles of projects, usefult for example for FetchProject dependencies. This would require two sets of rule variables and for cmake to select one dependening on the policy during generation or using the $<TARGET_POLICY:> generator expression. I don’t know how CMAKE_<LANG>_USING_LINKER_MODE could be handled, as

Scripts probably don’t expect a genex there.

  • If not set by the project when should CMP<XXX> emit a warning

  • Whenever IntelLLVM is selected: may be too noisy

  • When target_link_options is used: requires more effort and CMake maintainer buy-in

  • Do not warn in CMake, use OLD behaviour if unset. IntelSyclConfig may warn: users of new CMake with old OneAPI toolkits get no warning.

I would love to hear opinions and suggestions to the open questions. If this looks okay I can prepare a PR.

1 Like
2

Thanks for the detailed report and proposal.

There is nothing we can do about 3.25 through 3.30. Their behavior is now long established and is not going to change. We can’t backdate a policy or introduce one in patch releases. We also can’t make a policy that’s introduced in a new CMake version pretend that it was added in an older version. Also, policies are not behavior switches and should not be the only way to choose between two desirable behaviors.

To move forward, we need to approach this as if the current behavior were the way it always was and consider new controls/policies for future versions. If both forms are desirable, an option should be added to choose between them. If one of those is a better default, we can consider a policy to change the default of the new option.

IMO the 3.25+ behavior of linking through the compiler driver is more correct and shouldn’t change. That’s what’s done on all non-Windows platforms. I don’t think there is anything wrong with telling projects to use target_link_options with a LINKER: prefix. That’s the way we’ve established to tell CMake that an option is to be passed directly to the linker. CMake’s generators can then decide how to pass it through the compiler driver if necessary.

3

No backporting or pretending on policies makes sense, thanks.

I agree that in a vacuum the 3.25 behavior would be good, the main problem is that it creates compatibility issues if the Intel compiler behaves differently than clang-cl/msvc while traditionally being compatible and also setting CMAKE_<LANG>_COMPILER_FRONTEND_VARIANT to MSVC .

I don’t think there is anything wrong with telling projects to use target_link_options with a LINKER: prefix.

As a semi-serious idea, for the compilers that pass all flags to the linker (clang-cl, MSVC) could CMake require that the LINKER: prefix be used?

4

When generating a direct linker invocation perhaps we can try to detect if linker flags taken from LINK_OPTIONS or INTERFACE_LINK_OPTIONS did not come with a LINKER: prefix. If so, we could issue a warning that the project’s specification is not portable. It may take some investigation to determine if this is possible to detect reliably, and to choose appropriate wording and controls for the warning.

5

I can investigate if detection is can be made reliable.

For controls do you think there’s a need to opt-out of just this warning on a per-target level for example? The simplest I can think of is to raise an author warning, then -Wno-dev can be used to opt-out and -Werror=dev to turn it into an error.

6

I think making it an AUTHOR_WARNING such that -Wdev controls it is fine. We can always add more granular controls later.