# ctest complains about the test executable, but then deletes it?

**URL:** https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344
**Category:** Usage
**Tags:** os:linux, tool:ctest
**Created:** [March 12, 2024, 1:07pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344 "2024-03-12T13:07:32Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![dtgriscom](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/d/59ef9b/32.png) [@dtgriscom](https://discourse.cmake.org/u/dtgriscom)
#### Post date: [March 12, 2024, 1:07pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/1 "2024-03-12T13:07:32Z")

</div>

I’m taking over some code and trying to use it in a cross-platform compilation environment (building on Linux/x86, compiling to ARM). However, when I try to run the tests, ctest complains about the built executable and then immediately deletes it, making it awfully difficult to figure out what’s going wrong. (It’s likely being built for ARM and then run on x86, but I can’t tell.)

How can I keep cmake/ctest from deleting the executable?

Here’s what happens:

```auto
griscom@nob:~/git/nodes/_build$ ../run-docker make exmymod_tests
----snip----
[100%] Linking CXX executable exmymod_tests
/home/griscom/git/nodes/_build/MyCore/modules/mymod/exmymod_tests: 1: Syntax error: word unexpected (expecting ")")
CMake Error at /home/griscom/git/nodes/_build/_deps/catch2-src/extras/CatchAddTests.cmake:70 (message):
  Error running test executable
  '/home/griscom/git/nodes/_build/MyCore/modules/mymod/exmymod_tests':

    Result: 2
    Output: 

Call Stack (most recent call first):
  /home/griscom/git/nodes/_build/_deps/catch2-src/extras/CatchAddTests.cmake:175 (catch_discover_tests_impl)

make[3]: *** [MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/build.make:113: MyCore/modules/mymod/exmymod_tests] Error 1
make[3]: *** Deleting file 'MyCore/modules/mymod/exmymod_tests'
make[2]: *** [CMakeFiles/Makefile2:3734: MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:3741: MyCore/modules/mymod/CMakeFiles/exmymod_tests.dir/rule] Error 2
make: *** [Makefile:1226: exmymod_tests] Error 2
Cleaning up

```

Here’s the relevant section of the CMakeLists.txt:

```auto
# Add some tests
# temporary hack to get a properly functioning catch
Include(FetchContent)
FetchContent_Declare(
        Catch2
        GIT_REPOSITORY https://github.com/catchorg/Catch2.git
        GIT_TAG v3.4.0
)

FetchContent_MakeAvailable(Catch2)
#-------------------------- end of temporary hack

include(Catch)

add_executable(${PROJECT_NAME}_tests unittests/utilTests.cpp)
target_include_directories(${PROJECT_NAME}_tests PRIVATE src)
#target_sources(${PROJECT_NAME}_tests PRIVATE src/myMod.cpp)
target_link_libraries(${PROJECT_NAME}_tests
        PUBLIC
        ${TGT_LINK_LIBS}
        MyCore::${GENERATED_LIBRARY_TARGET}
        MyCore::exnewlogger
        Catch2::Catch2WithMain
)
catch_discover_tests(${PROJECT_NAME}_tests)

```

---

<div class="post-metadata">

### Author: ![buildSystemPerson](https://discourse.cmake.org/user_avatar/discourse.cmake.org/buildsystemperson/32/2851_2.png) [@buildSystemPerson](https://discourse.cmake.org/u/buildSystemPerson)
#### Post date: [March 12, 2024, 3:33pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/2 "2024-03-12T15:33:53Z")

</div>

> [@dtgriscom](#):
>
> `catch_discover_tests(${PROJECT_NAME}_tests)`

`catch_discover_tests` works by running the resulting executable with `--list-test-names-only` flag, and then parsing the output to find all existing tests.

The same principle applies to `gtest_discover_tests`.

So it will fail in cross-compiling scenarios unless [`CROSSCOMPILING_EMULATOR`](https://cmake.org/cmake/help/latest/prop_tgt/CROSSCOMPILING_EMULATOR.html#prop_tgt:CROSSCOMPILING_EMULATOR) is properly set.

[Catch](https://github.com/catchorg/Catch2/blob/devel/docs/cmake-integration.md#catchcmake-and-catchaddtestscmake) docs for reference.

---

<div class="post-metadata">

### Author: ![dtgriscom](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/d/59ef9b/32.png) [@dtgriscom](https://discourse.cmake.org/u/dtgriscom)
#### Post date: [March 12, 2024, 5:38pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/3 "2024-03-12T17:38:16Z")

</div>

Good information on cross-compilation: thank you.

The primary question stands, though. If I do a native build, then cmake/ctest leaves the (working) executable in place. But, if I do a cross-compile, and the executable is faulty, then cmake immediately deletes the executable, making it hard to diagnose the problem.

Why does cmake delete the executable? Is there any way to convince it to leave it in place for examination?

---

<div class="post-metadata">

### Author: ![buildSystemPerson](https://discourse.cmake.org/user_avatar/discourse.cmake.org/buildsystemperson/32/2851_2.png) [@buildSystemPerson](https://discourse.cmake.org/u/buildSystemPerson)
#### Post date: [March 13, 2024, 1:54am UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/4 "2024-03-13T01:54:47Z")

</div>

> Why does cmake delete the executable?

This is the default behaviour of make:

> <https://stackoverflow.com/questions/588550/why-does-gnu-make-delete-a-file>

---

<div class="post-metadata">

### Author: ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)
#### Post date: [March 13, 2024, 1:57am UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/5 "2024-03-13T01:57:20Z")

</div>

Place the following line before the call to `catch_discover_tests()`:

```cmake
set(CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE PRE_TEST)

```

This will defer test discovery to test time instead of doing it as a POST\_BUILD custom step. That has the advantage that it won’t interfere with building your test executable, which now won’t be deleted on you like it is now. It also ensures that test discovery happens on the same target architecture, etc. that your tests run on.

---

<div class="post-metadata">

### Author: ![dtgriscom](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/d/59ef9b/32.png) [@dtgriscom](https://discourse.cmake.org/u/dtgriscom)
#### Post date: [March 13, 2024, 2:10am UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/6 "2024-03-13T02:10:12Z")

</div>

Primo! That worked.

For future visitors, [here’s the doc](https://github.com/catchorg/Catch2/blob/devel/docs/cmake-integration.md#user-content-catch_discover_tests) on why this works:

Many thanks,  
Dan

---

<div class="post-metadata">

### Author: ![dtgriscom](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/d/59ef9b/32.png) [@dtgriscom](https://discourse.cmake.org/u/dtgriscom)
#### Post date: [March 13, 2024, 11:46am UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/7 "2024-03-13T11:46:20Z")

</div>

Although I’ve fixed the root cause, I still have the same question: how do I persuade cmake/ctest not to delete a test executable that has been improperly built? I’m sure this has happened for others, and is likely to happen again for me. Without the ability to examine that executable (or to consult with this forum’s experts) there’s no way I could figure out what the problem is.

---

<div class="post-metadata">

### Author: ![buildSystemPerson](https://discourse.cmake.org/user_avatar/discourse.cmake.org/buildsystemperson/32/2851_2.png) [@buildSystemPerson](https://discourse.cmake.org/u/buildSystemPerson)
#### Post date: [March 13, 2024, 3:29pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/8 "2024-03-13T15:29:06Z")

</div>

> [@dtgriscom](#):
>
> how do I persuade cmake/ctest not to delete a test executable that has been improperly built?

This is the default behaviour of `make`. When a command returns an error code (e.g. non-zero return) then the make target is deleted. Because the target might not have been built correctly. The next time you `make` the project, it will attempt to rebuild the target. If the file had not been removed, `make` would have no way of knowing something went wrong.

It does look like make has mechanisms to NOT delete make targets via the `.PRECIOUS` and `.IGNORE` makefile directives but those directives aren’t used by CMake at the moment.

However, testing locally I can verify this doesn’t occur with the `Ninja` generator. So that may be your quickest immediate solution.

---

<div class="post-metadata">

### Author: ![dtgriscom](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/d/59ef9b/32.png) [@dtgriscom](https://discourse.cmake.org/u/dtgriscom)
#### Post date: [March 13, 2024, 3:47pm UTC](https://discourse.cmake.org/t/ctest-complains-about-the-test-executable-but-then-deletes-it/10344/9 "2024-03-13T15:47:17Z")

</div>

So, in theory I could go through the cmake-generated `Makefile`s, find the `exmymod_tests` target, and hand-add that as a dependency to a `.PRECIOUS` target. Good info.

For future visitors, [here](https://www.gnu.org/software/make/manual/html_node/Special-Targets.html) is a list of Gnu Make special targets. (I’d known about `.PHONY`, but the others are new to me.)

Thanks,  
Dan