How to debug "Detecting C compiler ABI info" hanging (Cygwin on GitHub Actions)

My cmake builds on GitHub Actions with Cygwin have recently started timing out at 6 hours, when they used to take 10 minutes or so.

I’ve been looking at it, and the difference in output is:

last working build:

2021-10-27T01:22:00.2370158Z /home/runneradmin
2021-10-27T01:22:00.7132038Z -- The C compiler identification is GNU 11.2.0
2021-10-27T01:22:01.1511976Z -- The CXX compiler identification is GNU 11.2.0
2021-10-27T01:22:01.3345115Z -- Detecting C compiler ABI info
2021-10-27T01:22:01.4569110Z -- Detecting C compiler ABI info - failed
2021-10-27T01:22:01.4579158Z -- Check for working C compiler: /usr/bin/cc
2021-10-27T01:22:01.8781617Z -- Check for working C compiler: /usr/bin/cc - works

First failing build:

2021-10-31T11:02:09.8670109Z /home/runneradmin
2021-10-31T11:02:10.4093577Z -- The C compiler identification is GNU 11.2.0
2021-10-31T11:02:10.9970469Z -- The CXX compiler identification is GNU 11.2.0
2021-10-31T11:02:11.2962653Z -- Detecting C compiler ABI info
2021-10-31T16:58:56.5710031Z ##[error]The operation was canceled.
2021-10-31T16:58:56.6550538Z Post job cleanup.

So the Cmake step “Detecting C compiler ABI info” is never completing.

I’ve logged details in this issue in my project - with links to build logs:

Cmake is run with:

cmake -G 'Unix Makefiles' .. -DCMAKE_BUILD_TYPE=Release --log-level=VERBOSE

The output, in a branch where I have disabled most of the project to get faster feedback, currently is giving:

cmake version 3.20.0

CMake suite maintained and supported by Kitware (kitware.com/cmake).
-- The C compiler identification is GNU 11.2.0
-- The CXX compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
Error: The operation was canceled.

What can I do to debug this problem, or has anyone seen it and already fixed it, please?

This is how I would try to get more information if it was at my desk (not part of a CI build).

Delete the entire binary directory and start fresh. Then on the initial run use --debug-trycompile. This will keep all the project files for the ABI info and other tests.

Also the CMakeFiles/CMakeError.log file should capture the compiler output when the ABI test fails.

Thank you very much!

I added the --debug-trycompile argument, but didn’t get any extra console output.

What I don’t understand is how, in a GitHub Actions job where the Cmake command hangs until the entire job times out, I can can examine the contents of any log files?

I’m experimenting with timeout, to see if I can kill the cmake command and then print the logs

The errors you’d see would not get printed to any output for capture. They get put directly into the CMakeErrors.txt. You could try --trace-expand and similar options. If your system capture standard error than there is a good chance you’ll get details on what is happening. --trace-expand will show in great detail the commands leading up to the point of error.

1 Like

Firstly, thank you very much for the help and suggestions!

There is no CMakeErrors.txt file created.

Here’s the more detailed output that I’ve managed to get - extracted from the GitHub Actions build log:

2021-12-01T17:45:20.3495606Z /usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerABI.cmake(47):  set(ENV{LANG} C )
2021-12-01T17:45:20.3496681Z    Called from: [3]	/usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerABI.cmake
2021-12-01T17:45:20.3497634Z                 [2]	/usr/share/cmake-3.20.0/Modules/CMakeTestCCompiler.cmake
2021-12-01T17:45:20.3498487Z                 [1]	/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/CMakeLists.txt
2021-12-01T17:45:20.3501162Z /usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerABI.cmake(49):  try_compile(CMAKE_C_ABI_COMPILED /cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/cmake-build /usr/share/cmake-3.20.0/Modules/CMakeCCompilerABI.c CMAKE_FLAGS -DEXE_LINKER_FLAGS=-v;-DCMAKE_C_STANDARD_LIBRARIES= --no-warn-unused-cli COMPILE_DEFINITIONS -v OUTPUT_VARIABLE OUTPUT COPY_FILE /cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/cmake-build/CMakeFiles/3.20.0/CMakeDetermineCompilerABI_C.bin COPY_FILE_ERROR _copy_error __CMAKE_INTERNAL ABI )
2021-12-01T17:45:20.3504057Z    Called from: [3]	/usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerABI.cmake
2021-12-01T17:45:20.3504991Z                 [2]	/usr/share/cmake-3.20.0/Modules/CMakeTestCCompiler.cmake
2021-12-01T17:45:20.3505859Z                 [1]	/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/CMakeLists.txt
2021-12-01T17:45:20.3506596Z -- The CXX compiler identification is GNU 11.2.0
2021-12-01T17:45:20.3507424Z    Called from: [3]	/usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerId.cmake
2021-12-01T17:45:20.3508436Z                 [2]	/usr/share/cmake-3.20.0/Modules/CMakeDetermineCXXCompiler.cmake
2021-12-01T17:45:20.3509374Z                 [1]	/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/CMakeLists.txt
2021-12-01T17:45:20.3510045Z -- Detecting C compiler ABI info
2021-12-01T17:45:20.3510789Z    Called from: [3]	/usr/share/cmake-3.20.0/Modules/CMakeDetermineCompilerABI.cmake
2021-12-01T17:45:20.3511725Z                 [2]	/usr/share/cmake-3.20.0/Modules/CMakeTestCCompiler.cmake
2021-12-01T17:45:20.3512579Z                 [1]	/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/CMakeLists.txt
2021-12-01T17:45:27.3809111Z Timed out
2021-12-01T17:45:27.3810346Z ------------------------------------------------------------------------------------------------
2021-12-01T17:45:27.3811407Z executing: ls -ls
2021-12-01T17:45:27.4017530Z total 0
2021-12-01T17:45:27.4018900Z 0 drwxrwxr-x+ 1 runneradmin None 0 Dec  1 17:45 CMakeFiles
2021-12-01T17:45:27.4024270Z ------------------------------------------------------------------------------------------------
2021-12-01T17:45:27.4029426Z executing: ls -ls CMakeFiles/
2021-12-01T17:45:27.4225838Z total 12
2021-12-01T17:45:27.4227573Z 4 drwxrwxr-x+ 1 runneradmin None   0 Dec  1 17:45 3.20.0
2021-12-01T17:45:27.4229164Z 4 -rw-rw-r--+ 1 runneradmin None 861 Dec  1 17:45 CMakeOutput.log
2021-12-01T17:45:27.4230108Z 4 drwxrwxr-x+ 1 runneradmin None   0 Dec  1 17:45 CMakeTmp
2021-12-01T17:45:27.4230905Z ------------------------------------------------------------------------------------------------
2021-12-01T17:45:27.4240126Z executing: cat CMakeFiles/CMakeError.log
2021-12-01T17:45:27.4408741Z cat: CMakeFiles/CMakeError.log: No such file or directory
2021-12-01T17:45:27.4416542Z Cannot cat CMakeFiles/CMakeError.log
2021-12-01T17:45:27.4424273Z ------------------------------------------------------------------------------------------------
2021-12-01T17:45:27.4425612Z executing: cat CMakeFiles/CMakeFiles/CMakeOutput.log
2021-12-01T17:45:27.4570327Z The system is: CYGWIN - 3.3.2(0.341/5/3) - x86_64
2021-12-01T17:45:27.4571683Z Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
2021-12-01T17:45:27.4573297Z Compiler: /usr/bin/cc.exe 
2021-12-01T17:45:27.4576211Z Build flags: 
2021-12-01T17:45:27.4579178Z Id flags:  
2021-12-01T17:45:27.4580291Z 
2021-12-01T17:45:27.4585383Z The output was:
2021-12-01T17:45:27.4588410Z 0
2021-12-01T17:45:27.4588756Z 
2021-12-01T17:45:27.4589054Z 
2021-12-01T17:45:27.4589772Z Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "a.exe"
2021-12-01T17:45:27.4590414Z 
2021-12-01T17:45:27.4591367Z The C compiler identification is GNU, found in "/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/cmake-build/CMakeFiles/3.20.0/CompilerIdC/a.exe"
2021-12-01T17:45:27.4592218Z 
2021-12-01T17:45:27.4593053Z Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" succeeded.
2021-12-01T17:45:27.4593885Z Compiler: /usr/bin/c++.exe 
2021-12-01T17:45:27.4594355Z Build flags: 
2021-12-01T17:45:27.4594946Z Id flags:  
2021-12-01T17:45:27.4595253Z 
2021-12-01T17:45:27.4595659Z The output was:
2021-12-01T17:45:27.4596038Z 0
2021-12-01T17:45:27.4596319Z 
2021-12-01T17:45:27.4596598Z 
2021-12-01T17:45:27.4597366Z Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "a.exe"
2021-12-01T17:45:27.4598063Z 
2021-12-01T17:45:27.4599011Z The CXX compiler identification is GNU, found in "/cygdrive/d/a/ApprovalTests.cpp/ApprovalTests.cpp/cmake-build/CMakeFiles/3.20.0/CompilerIdCXX/a.exe"
2021-12-01T17:45:27.4599879Z 
2021-12-01T17:45:27.4601530Z ------------------------------------------------------------------------------------------------

It looks like CMake is hanging in the middle of the try_compile at 2021-12-01T17:45:20.3501162Z.

What’s confusing to me is the Called from: lines from 2021-12-01T17:45:20.3507424Z and later.
They don’t show what CMake command is being executed! The output -- The CXX compiler identification is GNU 11.2.0 and -- Detecting C compiler ABI info are from message commands well before the try_compile command.

When try_compile runs its given an infinite timeout when spawning the build tool to build the project. I think the build tool console output gets redirected back to CMake so we cannot see what goes wrong.

The timeout 10 cmake ... invocation prints the “Timed out” message.

But this explains why you don’t get a CMakeError.log file. CMake doesn’t generate an error until after returning from a failed try_compile.

@craig.scott Any suggestions?

All that being said. If you use find you should see the project that CMake generates for the try_compile. You should be able to run make in that directory and see if it can build the project or if make is the program that is hanging.

Brilliant idea - I’ll add steps to list the contents of the directory where try_compile is run, and go from there…