Help with CC, Clang, Environment, and CMake

All,

I think I might be an idiot, but I’m not sure what’s going on. I’m having issues with CMake because it seems to be finding the “wrong clang”. In an autotools build (controlled by CMake/ExternalProject_Add), it’s trying to run a program that is essentially:

#include <stdio.h>

int main() {
  printf("Hello world\n");
}

and it’s failing. Why? Because it’s calling the “wrong” clang. This works:

❯ /usr/bin/clang -o conftest -fPIC test.c
❯ echo $?
0

This does not:

❯ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -o conftest -fPIC test.c
test.c:1:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^~~~~~~~~
1 error generated.
❯ echo $?
1

So, I start drilling down and I see some oddness. I made up this CMakeLists.txt:

cmake_minimum_required(VERSION 3.17)
project(test VERSION 1.0.0 LANGUAGES C)
message(STATUS "CC from ENV: $ENV{CC}")
set(SERIAL_C_COMPILER ${CMAKE_C_COMPILER})
message(STATUS "SERIAL_C_COMPILER: ${SERIAL_C_COMPILER}")

Now I try and use it. First I load my module and echo out CC as set by the modulefile:

❯ ml purge
❯ ml intel-clang
❯ echo $CC
/usr/bin/clang

Now I run cmake:

❯ cmake ..
-- The C compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- CC from ENV: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- SERIAL_C_COMPILER: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/mathomp4/CmakeTest/build

Instead of using the CC from my environment, it’s using…a different one.

So is there a way for CMake to use what I have in CC for determining the C compiler? Or do I just always need to pass in -DCMAKE_C_COMPILER=$CC which seems to work:

❯ cmake .. -DCMAKE_C_COMPILER=$CC
-- The C compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- CC from ENV: /usr/bin/clang
-- SERIAL_C_COMPILER: /usr/bin/clang
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/mathomp4/CmakeTest/build

to have CMake use the CC in my environment?

Or is there a way to tell CMake don’t use the /deep/inside/xcode/usr/bin/clang but actually /usr/bin/clang?

My guess is that you’ve install Xcode, but haven’t run it. Once run, Xcode installs some things so that the command line tools work as intended.

@ben.boeckel No, I have Xcode and Xcode Command Line utilities installed. And if I use autotools outside of CMake (where it picks up $CC from the environment) it works just fine. It’s somehow CMake itself that is doing this resolving…

Probably worth looking at what which cmake prints and investigate what it is that you are actually running when invoking cmake in your shell.

Some tracing of how the compiler is determined would be nice. --trace-expand might be able to shed some light on the CMake code portion of that at least.

Cc: @brad.king

Here is the CMake info:

❯ which cmake
/Users/mathomp4/.homebrew/brew/bin/cmake
❯ cmake --version
cmake version 3.19.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).

And I’m attaching the output of:

cmake .. --trace-expand |& tee trace.out

trace.out (1.5 MB)

This thread on StackOverflow may be of interest to you

@vre Unfortunately, I did not build Clang myself. I’m using the Clang from XCode. And, just like C, I see the same issue with C++. If I expand my CMakeLists.txt:

cmake_minimum_required(VERSION 3.17)
project(test VERSION 1.0.0 LANGUAGES C CXX)
message(STATUS "CC from ENV: $ENV{CC}")
set(SERIAL_C_COMPILER       ${CMAKE_C_COMPILER})
message(STATUS "SERIAL_C_COMPILER: ${SERIAL_C_COMPILER}")
message(STATUS "CXX from ENV: $ENV{CXX}")
set(SERIAL_CXX_COMPILER       ${CMAKE_CXX_COMPILER})
message(STATUS "SERIAL_CXX_COMPILER: ${SERIAL_CXX_COMPILER}")
❯ echo $CC $CXX
/usr/bin/clang /usr/bin/clang++
❯ cmake ..
-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CC from ENV: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- SERIAL_C_COMPILER: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-- CXX from ENV: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
-- SERIAL_CXX_COMPILER: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/mathomp4/CmakeTest/build

So, again, CMake is seeing the “long” CXX. And if I try a tester:

#include <ostream>
#include <algorithm>

int main(int argc, char** argv) {
return 0;
}
❯ /usr/bin/clang++ -o conftest ./test.cc
❯ echo $?
0
❯ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o conftest ./test.cc
In file included from ./test.cc:1:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/ostream:138:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/ios:214:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iosfwd:95:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/wchar.h:118:15: fatal error: 'wchar.h' file not
      found
#include_next <wchar.h>
              ^~~~~~~~~
1 error generated.
❯ echo $?
1

See CMake Issue 19180.

Currently CMake unwraps /usr/bin/clang to be the full path to the Clang inside the currently active Xcode. When launching the compiler without the wrapper, an explicit -isysroot /path/to/macOS/SDK is needed. CMake adds this, but other buildsystems may not.

A compounding factor is that CMake also sets the CC and CXX environment variables to match whatever compiler it resolved so that other configure scripts executed during the configuration via execute_process will see it. This historical behavior may be removed with a policy. See CMake Issue 21378.

Hmm. Interesting. Well, I’ll see if I can figure out a way to inject it into the sub-autotools build.

As for the policy, do you know what the number is? I tried reading man 7 cmake-policies but I didn’t see it (might have got my search wrong).

There isn’t such a policy yet. The linked issue discusses it.

Ohhh. Well I feel dumb. I thought that issue was referring to a policy already around. :man_facepalming: