CMake 3.21 rc-1 | HIP support breaks custom MSVC toolchain

buildSystemPerson · June 25, 2021, 9:13pm

Since I’ve upgraded to CMake 3.21 our internal MSVC toolchains have broken.

It’s not longer able to identify the compiler. It’s no longer able to detect the CXX compiler ABI info. As a result it seems to be unable to produce a simple test program.

I’m using Ninja as the generator and I’m using MSVC version 19.28.29910.

My toolchain file for MSVC is pretty minimal. I’m setting the

CMAKE_SYSTEM_PROCESSOR
CMAKE_CXX_COMPILER
CMAKE_C_COMPILER
CMAKE_RC_COMPILER
CMAKE_MT

Here is the error message:

-- The CXX compiler identification is unknown
-- The C compiler identification is unknown
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler: d:/dk/win/vc/14.28.29910/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: d:/dk/win/vc/14.28.29910/bin/Hostx64/x64/cl.exe - broken
CMake Error at C:/CMake/share/cmake-3.21/Modules/CMakeTestCXXCompiler.cmake:59 (message):
  The C++ compiler

    "d:/dk/win/vc/14.28.29910/bin/Hostx64/x64/cl.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: D:/github/foobar_cmake_modules/tests/test0/build/nin/CMakeFiles/CMakeTmp
    
    Run Build Command(s):D:/dk/win/ninja/1.10.2/ninja.exe cmTC_1f78e && [1/2] Building CXX object CMakeFiles\cmTC_1f78e.dir\testCXXCompiler.cxx.obj
    Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
    testCXXCompiler.cxx
    [2/2] Linking CXX executable cmTC_1f78e.exe
    FAILED: cmTC_1f78e.exe 
    cmd.exe /C "cd . && d:\dk\win\vc\14.28.29910\bin\Hostx64\x64\cl.exe /FS /LIBPATH:d:/dk/win/vc/14.28.29910/lib/x64 /LIBPATH:d:/dk/win/vc/14.28.29910/atlmfc/lib/x64 /LIBPATH:d:/dk/win/ms_sdk/n21322/10/lib/10.0.21322.0/ucrt/x64 /LIBPATH:d:/dk/win/ms_sdk/n21322/10/lib/10.0.21322.0/um/x64 /LIBPATH:d:/dk/win/ms_wdk/n21322/lib/10.0.21322.0/km/x64 /LIBPATH:d:/dk/win/ms_wdk/n21322/lib/10.0.21322.0/um/x64 CMakeFiles\cmTC_1f78e.dir\testCXXCompiler.cxx.obj -o cmTC_1f78e.exe   && cd ."
    Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    cl : Command line warning D9035 : option 'o' has been deprecated and will be removed in a future release
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/vc/14.28.29910/lib/x64'
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/vc/14.28.29910/atlmfc/lib/x64'
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/ms_sdk/n21322/10/lib/10.0.21322.0/ucrt/x64'
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/ms_sdk/n21322/10/lib/10.0.21322.0/um/x64'  
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/ms_wdk/n21322/lib/10.0.21322.0/km/x64'     
    cl : Command line warning D9002 : ignoring unknown option '/LIBPATH:d:/dk/win/ms_wdk/n21322/lib/10.0.21322.0/um/x64'
    Microsoft (R) Incremental Linker Version 14.28.29910.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    /out:testCXXCompiler.cxx.exe 
    /out:cmTC_1f78e.exe
    CMakeFiles\cmTC_1f78e.dir\testCXXCompiler.cxx.obj
    LINK : fatal error LNK1181: cannot open input file 'CMakeFiles\cmTC_1f78e.dir\testCXXCompiler.cxx.obj'
    ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:22 (project)


-- Configuring incomplete, errors occurred!
See also "D:/github/foobar_cmake_modules/tests/test0/build/nin/CMakeFiles/CMakeOutput.log".
See also "D:/github/foobar_cmake_modules/tests/test0/build/nin/CMakeFiles/CMakeError.log".

ben.boeckel · June 25, 2021, 9:32pm

Hmm. This is quite odd. We do have CI running with this setup, though we’ve moved onto a newer compiler (14.29.30037). Would you be willing and able to bisect the problem?

Looking at your setup, it seems you’re using a different location for the compiler than is normal. There were some changes for “relocatable VS toolchains”, but I don’t the change(s) involved that well.

Cc: @brad.king

buildSystemPerson · June 25, 2021, 9:37pm

That is correct.

My company basically doesn’t rely on the MSVC compilers provided by Visual Studio. So we host the binaries in a separate location.

ben.boeckel · June 25, 2021, 9:41pm

Hmm. I can’t find the MR(s) right now, but if you do have a chance to bisect, searching for where CMake 3.21 rc-1 | Threads::Threads not found got introduced would also be handy.

buildSystemPerson · June 25, 2021, 9:44pm

Also I tried using a newer/older compiler and it doesn’t seem to matter at all. I still get the same error.

buildSystemPerson · June 25, 2021, 10:22pm

EDIT:
I think I can figure it out actually.

buildSystemPerson · June 28, 2021, 5:16pm

I was having issues doing commit bisects but I did narrow the issue down to a change in CMakeDetermineCompilerId.cmake

buildSystemPerson · June 28, 2021, 5:33pm

CMakeCxxCompilerId.cmake fails to compile in 3.21 due to not being able to find hip/hip_version.h

It seems to fail a execute_process call due to the lack of this hip/hip_version.h which in turn doesn’t allow the CMake logic to determine the compiler ID causing everything else to fail…

Is there a way to turn HIP off?

ben.boeckel · June 28, 2021, 5:38pm

Cc: @robert.maynard

buildSystemPerson · June 28, 2021, 6:09pm

Here is the compiler error:

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: d:/dk/win/vc/14.28.29910/bin/Hostx64/x64/cl.exe 
Build flags: /FS
Id flags: -c 

The output was:
2
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29910 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

CMakeCXXCompilerId.cpp
CMakeCXXCompilerId.cpp(304): fatal error C1034: hip/hip_version.h: no include path set

And here is the file:
CMakeCXXCompilerId.cpp (24.9 KB)

This is where it fails to compiler:

// Fails to compile on this line.... I'm not sure why though...
#elif defined(__clang__) && __has_include(<hip/hip_version.h>)
# define COMPILER_ID "ROCMClang"
# if defined(_MSC_VER)
#  define SIMULATE_ID "MSVC"
# elif defined(__clang__)
#  define SIMULATE_ID "Clang"
# elif defined(__GNUC__)
#  define SIMULATE_ID "GNU"
# endif
# if defined(__clang__) && __has_include(<hip/hip_version.h>)
#  include <hip/hip_version.h>
#  define COMPILER_VERSION_MAJOR DEC(HIP_VERSION_MAJOR)
#  define COMPILER_VERSION_MINOR DEC(HIP_VERSION_MINOR)
#  define COMPILER_VERSION_PATCH DEC(HIP_VERSION_PATCH)
# endif

robert.maynard · June 28, 2021, 6:11pm

@zackgalbreath

Looking at the compiler detection that would only occur if __has_include() returns but the header doesn’t actually exist.

I also don’t think that compiler is supported on windows, so we should drop it from the detection list

buildSystemPerson · June 28, 2021, 6:26pm

You are correct I really don’t understand why this is happening. The logic looks fine to me. Something is happening though to cause it to evaluate to true and try including the HIP header…

I’ll try and see if I can dig any further.

But since HIP currently doesn’t support Windows, I’d appreciate it if you guys dropped it from the detection list for now.

buildSystemPerson · June 28, 2021, 6:36pm

Just for reference I tried using the compiler directly and it didn’t work either…

buildSystemPerson · June 28, 2021, 6:56pm

I think I figured it out.

In order for __has_include to work properly on MSVC it requires at least 1 include directory. Since I’m using a toolchain for my toolchain environment and not relying on the developer command prompts it was always failing.

I cheated and just used the directory that CMakeCXXCompilerId.cpp resides in:

EDIT:

I just confirmed my theory on MSVC __has_include. It absolutely requires at least one include directory. If I add 1 include directory to my toolchain file it works.

buildSystemPerson · June 28, 2021, 7:10pm

Basically to summarize this issue here is what happened.

3.21 introduced HIP.
HIP’s detection relies on __has_include
MSVC’s __has_include relies on the user having valid include directories set. Either by the environment/command line.
As a result my toolchain broke despite the fact I was using CMAKE_LANG_STANDARD_INCLUDE_DIRECTORIES
I can fix the problem on my side by adding a include directory to the CMAKE_LANG_FLAGS_INIT variable.

What this means is that CMAKE_LANG_STANDARD_INCLUDE_DIRECTORIES isn’t taken into account when compiling the CMakeCXXCompiler.* files. Perhaps it should though?

ben.boeckel · June 28, 2021, 7:28pm

How can it? CMakeCXXCompiler.* files are for CMake to figure out what compiler it is dealing with in the first place, so CMake would need to know how to extract standard include directories from a compiler it doesn’t know how to invoke yet (because it doesn’t know the flavor of it)?

buildSystemPerson · June 28, 2021, 7:33pm

Well I guess in my situation I’m using a toolchain file. But obviously not everyone is using a toolchain file like I am.

Do you think adding a dummy include directory is out of the question? Because all MSVC needs is a single include directory to correctly give the expected behavior for __has_include

Would adding the directory of CMakeCXXCompilerId.cpp be out of the question? As shown above?

ben.boeckel · June 28, 2021, 7:53pm

I think it’s better to just guard the __has_include stuff for HIP against MSVC (since it’s obviously not a HIP compiler).

buildSystemPerson · June 28, 2021, 7:54pm

Fair enough

brad.king · June 29, 2021, 2:08pm

just guard the __has_include stuff for HIP against MSVC

That’s fine with me, but how is the defined(__clang__) part of the condition passing on MSVC?

My company basically doesn’t rely on the MSVC compilers provided by Visual Studio.

Are you using a Clang-derived alternative? What preprocessor definition should be checked to exclude the compiler?