Building and running one test

Usage
1

I am new to CMake and a bit confused.

I can do this to build and run all of my tests: cmake --build _builds --target test

I can do this to build one of my tests: cmake --build _builds --target some-test

Is there a way to build and run a specific test, or perhaps a set of tests by some pattern (name patterns, etc.)?

Since cmake creates a ‘test’ pseudotarget I’m looking for a corresponding per-test pseudotarget like test:mytest that would build and run mytest.

I have attempted to understand what CTest is, what its build-and-test mode is (which sounds a bit like what I want?), etc. Its feature that reruns only failed tests looks interesting. But I am confused because most of the terminology is unfamiliar.

For background, I am used to Google’s bazel, where I can acomplish what I want with its build and test subcommands:

bazel build <targets>
bazel test <targets>

I’m also not using an IDE. All my cmake use is from the command line or by way of Emacs.

2

CMake does the build, but CTest does the testing. There’s no real “integration” at build time other than the test target that just calls ctest. The best that is available is ctest -R <regex>, but this doesn’t guarantee that anything used by those tests are up-to-date.

CTest doesn’t know about build targets and the generated build code doesn’t know how to run specific tests. This is because a test may have other properties set that cannot be fulfilled by the build tool. Some such items (though not exhaustive):

  • WILL_FAIL: needs a wrapper command to detect failure and return success
  • {PASS,FAIL,SKIP}_REGULAR_EXPRESSION: needs a wrapper command to read the output and handle the state determination
  • SKIP_RETURN_CODE: what does it mean to skip via a build command?
  • ENVIRONMENT: also needs a wrapper
  • TIMEOUT: how will the build tool end a test after the timeout elapses?
  • REQUIRED_FIXTURES: build tools generally don’t support starting a command, holding it open, then running another command, and finally bringing down the first when the second completes

--build-and-test is mostly a mechanism to run a CTest suite given a CMake-using source tree with a single command. It’s not really what you’re looking for (I’ve mostly only seen it used in test suites involving projects).

3

Thanks Ben. Let me ask then, what do people do when they’d like to run one of many tests?

Assume I have the following in a CMakeLists.txt somewhere in a larger project:

add_library(example example.c)
add_executable(example_test example_test.c)
target_link_libraries(example_test PRIVATE example)
add_test(NAME example_test COMMAND example_test)

I then run cmake --build _builds --target test and find that, among other failures, example_test fails. It is a fast test to build and run so I want to focus on its failure first. What is the simplest cmake and/or ctest invocation (or both) that I can use to iterate on an edit/compile/link/test cycle for example_test alone?

The best I can come up with is unwieldy (expressed in Unix sh)

cmake --build _builds --target example_test && (cd _builds && ctest -R example_test)

…but if that is the best way I can easily script it.

Last evening I found a post here (which I can’t find now, sadly) that said something like “I used one of the well known techniques to deal with the fact that ctest can’t rebuild targets” which suggests that projects have come up with solutions to this problem. Are they discussed or documented anywhere?

4

Something like this:

$ $EDITOR files_to_fix.c
$ cmake --build example_test # the test binary
$ ctest -R example_test
$ ctest --rerun-failed # also available if you're whittling down a larger set of tests

which is largely covered by your shell snippet.

I don’t know of it, sorry.

1 Like
5

Thanks Ben, that helps!