CTEST_CUSTOM_TESTS_IGNORE doesn't use regex like docs suggest?

Usage
1

I think there’s a documentation bug for CTEST_CUSTOM_TESTS_IGNORE. The documentation says it uses regular expressions, but this does not seem to be the case.

ctest -N
Test project /home/vagrant/Projects/cxxscratch/build/GCC-Debug
  Test #1: CxxMain
  Test #2: CxxMain.memcheck
  Test #3: CMain

Total Tests: 3

When I create a ${CMAKE_BINARY_DIR}/CTestCustom.cmake with the following content

set(CTEST_CUSTOM_TESTS_IGNORE
  ".*CxxMain.*"
)

and run cmake --build . --target test, the CxxMain tests are still executed

cmake --build . --target test
[0/1] Running tests...
Test project /home/vagrant/Documents/cxxscratch/build/GCC-Debug
    Start 1: CxxMain
1/3 Test #1: CxxMain ..........................   Passed    0.00 sec
    Start 2: CxxMain.memcheck
2/3 Test #2: CxxMain.memcheck .................   Passed    0.01 sec
    Start 3: CMain
3/3 Test #3: CMain ............................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 3

Total Test time (real) =   0.02 sec

It doesn’t seem to accept GLOBs either. It does work if you match the test names explicitly. I personally would gladly welcome a change to use regex, but the docs at least should maybe be updated. Thanks!

2

I took a very quick look at the source code and you are correct. It just does a simple string comparison, it doesn’t treat it as a regular expression. Could you please report this issue in the issue tracker? In the first instance, the docs should be updated. As for actually making this support regular expressions, that might be tricky because of the need to preserve backward compatible behavior.

3

Thanks Craig!

I added a bug ticket here:
https://gitlab.kitware.com/cmake/cmake/-/issues/22758

Let me know if I should do anything else.
Thanks!

P.S. love the book by the way :slight_smile:

1 Like
4

A new variable name should suffice (CTEST_CUSTOM_TESTS_IGNORE_REGEX).