ctest complains about the test executable, but then deletes it?

Usage
,
1

I’m taking over some code and trying to use it in a cross-platform compilation environment (building on Linux/x86, compiling to ARM). However, when I try to run the tests, ctest complains about the built executable and then immediately deletes it, making it awfully difficult to figure out what’s going wrong. (It’s likely being built for ARM and then run on x86, but I can’t tell.)

How can I keep cmake/ctest from deleting the executable?

Here’s what happens:

griscom@nob:~/git/nodes/_build$ ../run-docker make exmymod_tests
----snip----
[100%] Linking CXX executable exmymod_tests
/home/griscom/git/nodes/_build/MyCore/modules/mymod/exmymod_tests: 1: Syntax error: word unexpected (expecting ")")
CMake Error at /home/griscom/git/nodes/_build/_deps/catch2-src/extras/CatchAddTests.cmake:70 (message):
  Error running test executable
  '/home/griscom/git/nodes/_build/MyCore/modules/mymod/exmymod_tests':


    Result: 2
    Output: 

Call Stack (most recent call first):
  /home/griscom/git/nodes/_build/_deps/catch2-src/extras/CatchAddTests.cmake:175 (catch_discover_tests_impl)


make[3]: *** [MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/build.make:113: MyCore/modules/mymod/exmymod_tests] Error 1
make[3]: *** Deleting file 'MyCore/modules/mymod/exmymod_tests'
make[2]: *** [CMakeFiles/Makefile2:3734: MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:3741: MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/rule] Error 2
make: *** [Makefile:1226: exmymod_tests] Error 2
Cleaning up

Here’s the relevant section of the CMakeLists.txt:

# Add some tests
# temporary hack to get a properly functioning catch
Include(FetchContent)
FetchContent_Declare(
        Catch2
        GIT_REPOSITORY https://github.com/catchorg/Catch2.git
        GIT_TAG        v3.4.0
)

FetchContent_MakeAvailable(Catch2)
#-------------------------- end of temporary hack

include(Catch)

add_executable(${PROJECT_NAME}_tests unittests/utilTests.cpp)
target_include_directories(${PROJECT_NAME}_tests PRIVATE src)
#target_sources(${PROJECT_NAME}_tests PRIVATE src/myMod.cpp)
target_link_libraries(${PROJECT_NAME}_tests
        PUBLIC
        ${TGT_LINK_LIBS}
        MyCore::${GENERATED_LIBRARY_TARGET}
        MyCore::exnewlogger
        Catch2::Catch2WithMain
)
catch_discover_tests(${PROJECT_NAME}_tests)
2

catch_discover_tests works by running the resulting executable with --list-test-names-only flag, and then parsing the output to find all existing tests.

The same principle applies to gtest_discover_tests.

So it will fail in cross-compiling scenarios unless CROSSCOMPILING_EMULATOR is properly set.

Catch docs for reference.

3

Good information on cross-compilation: thank you.

The primary question stands, though. If I do a native build, then cmake/ctest leaves the (working) executable in place. But, if I do a cross-compile, and the executable is faulty, then cmake immediately deletes the executable, making it hard to diagnose the problem.

Why does cmake delete the executable? Is there any way to convince it to leave it in place for examination?

4

Why does cmake delete the executable?

This is the default behaviour of make:

5

Place the following line before the call to catch_discover_tests():

set(CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE PRE_TEST)

This will defer test discovery to test time instead of doing it as a POST_BUILD custom step. That has the advantage that it won’t interfere with building your test executable, which now won’t be deleted on you like it is now. It also ensures that test discovery happens on the same target architecture, etc. that your tests run on.

1 Like
6

Primo! That worked.

For future visitors, here’s the doc on why this works:

Many thanks,
Dan

7

Although I’ve fixed the root cause, I still have the same question: how do I persuade cmake/ctest not to delete a test executable that has been improperly built? I’m sure this has happened for others, and is likely to happen again for me. Without the ability to examine that executable (or to consult with this forum’s experts) there’s no way I could figure out what the problem is.

8

This is the default behaviour of make. When a command returns an error code (e.g. non-zero return) then the make target is deleted. Because the target might not have been built correctly. The next time you make the project, it will attempt to rebuild the target. If the file had not been removed, make would have no way of knowing something went wrong.

It does look like make has mechanisms to NOT delete make targets via the .PRECIOUS and .IGNORE makefile directives but those directives aren’t used by CMake at the moment.

However, testing locally I can verify this doesn’t occur with the Ninja generator. So that may be your quickest immediate solution.

9

So, in theory I could go through the cmake-generated Makefiles, find the exmymod_tests target, and hand-add that as a dependency to a .PRECIOUS target. Good info.

For future visitors, here is a list of Gnu Make special targets. (I’d known about .PHONY, but the others are new to me.)

Thanks,
Dan