ctest "-I" does not behave as advertised in "Mastering CMake » Testing With CMake and CTest » Testing Using CTest"

This post is provided as an informative one rather than a plea for help.

I’ve successfully built my project using CMake; however, the CTest command line does behave as advertised when using the -I option (reference: Mastering CMake » Testing With CMake and CTest » Testing Using CTest). The -I option is intriguing for selective test execution, and it is documented as follows:

-I [Start,End,Stride,test#,test#|Test file]
                      Run specific tests by range and number.

Specifically, the following fails to limit execution to tests 12 and 21 in my suite:

ctest -j`corecount` --output-on-failure -I ,,,12,21

The following successfully executes the specific tests for CTest 3.26.3:

ctest -j`corecount` --output-on-failure -I 0,0,0,12,21

Neither of the commands listed above work for CTest 3.20.2.

These sort of behavioral and documentation inconsistencies provide a painful user experience. What is a humble(?) developer to do?

SOLUTION?

So far, the following command syntax resolves these behavioral discrepancies for varied versions of CTest:

ctest -j`corecount` --output-on-failure -I 0,0,1,12,21

It seems the Stride parameter is required to be non-zero and the Start,End values must not be blank for consistency when one wants to limit test execution to specific test numbers. It’s unfortunate that the so-called “CMake bible” does not clarify such nuance.

EDIT

One final comment to add to this post. The -LE option is incompatible with the -I option. Using them simultaneously may not execute the desired tests pecified by the -I option.

@bill.hoffman What’s our process for updating “Mastering CMake”?

It is open source now, so an MR would be the way to make changes.

For completion’s sake, the repo is here: https://gitlab.kitware.com/cmake/mastering-cmake