print list of tests via TEST_LIST variable

Usage
,
1

Could you please advise how view/print list of tests discovered by gtest_discover_tests. As per 3.17.5 user guide the option TEST_LIST var Make the list of tests available in the variable var… Note that this variable is only available in CTest."

gtest_discover_tests(lib_gtest
        WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}   
        TEST_LIST lib_test_list
        PROPERTIES ENVIRONMENT 
                     MYCONFIG_PATH=${CMAKE_CURRENT_SOURCE_DIR}
)

What does it mean “available in CTest”? is there example to print it?
The way ctest is executed is as the following template:

add_custom_command(TARGET lib_gtest
        POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E env ... ctest -j4 --output-on-failure --gtest_output=xml:lib_gtest_report.xml
        ....
        )

So could you please advise on:

  1. How to print TEST_LIST variable lib_test_list ?

  2. What does this mean " gtest_discover_tests sets up a post-build command on the test executable that generates the list of tests by parsing the output from running the test with the --gtest_list_tests argument" ? Is --gtest_list_tests argument to ctest needed?

  3. What does this mean " Additionally, setting properties on tests is somewhat less convenient, since the tests are not available at CMake time. Additional test properties may be assigned to the set of tests as a whole using the PROPERTIES option." - does it mean set_tests_properties() wont work on discovered tests? I tried it and got error that test is not found.

  4. What is proper use of PROPERTIES ENVIRONMENT ? Is it really parameter value like MYCONFIG_PATH ${NEWPATH} or parameter=value “MYCONFIG_PATH=${NEWPATH}”? , see googletest - With CMake, how can I set environment properties on the gtest_discover_tests --gtest_list_tests call? - Stack Overflow

. Note, OS Linux, cmake 3.17

2

You need to create a custom ctest file, which will be processed when ctest runs. It uses the same syntax language as CMake, but it is run at test time, not at configure time. Put something like this in your CMakeLists.txt file to tell CMake to process it at test time:

set_property(DIRECTORY APPEND PROPERTY
      TEST_INCLUDE_FILES ${CMAKE_CURRENT_LIST_DIR}/customTestManip.cmake
)

Then in customTestManip.cmake, you’d have code like this:

message(STATUS "List of tests: ${lib_test_list}")

The above is untested, but should be in the ballpark.

No you don’t need to add --gtest_list_tests. That sentence is just telling you what the command does internally.

3

Thank you for quick response.

Does this mean that add_custom_command(TARGET lib_gtest
POST_BUILD …
better be removed as well and all ctest processing be concentrated in customTestManip.cmake ?
4. Could you please respond to item 4. above?
I tried the above solution and it did not produce any list. However I do see a file created by ctest:
${CMAKE_BINARY_DIR}/Testing/Temporary/CTestCostData.txt This file stores data for each test in a row. So effectively it provides list of tests. However this list has been created without suggested change to set TEST_INCLUDE_FILES.
I still want to get your advise on printing this list but I dont like using non-target specific command like set_property so can this be done with set_target_properties or set_tests_properties? The latter seems does not work with gtest_discover_tests as I am getting no tests found error message.

Therefore the bigger question is how to set properties (e.g. environment variables to pass to test .cpp etc. ) if test are discovered with gtest_discover_tests?

4

Yes

Opening a big can of worms! This is also mostly why I didn’t have time to respond to this item in my earlier reply. :wink: First point, there’s no easy way to pass through a property that contains a value that is a list. There are multiple levels of CMake parsing involved, which results in making it unreliable to try to preserve the list. This is what the stackoverflow issue you linked to is mostly about, but I wouldn’t recommend the approach given in that answer. I’ll refer you to the following recently submitted issue for a deeper discussion, very relevant to your problem:

https://gitlab.kitware.com/cmake/cmake/-/issues/23329

As recommended in that issue, take a look at the ENVIRONMENT_MODIFICATION test property, which became available in CMake 3.22.

But ultimately, I think you’re still in trouble. Here’s another issue which I think captures the crux of the problem you’re facing:

https://gitlab.kitware.com/cmake/cmake/-/issues/21453

In short, test discovery on Windows is a real bummer when the target needs to find DLLs at discovery time.

5

Thank you Sir. Luckily these 2 links refers to complications which I might not affected by. I need to pass single (not a list) tuple variable-value for single path MYCONFIG_PATH string from cmake PROPERTIES ENVIRONMENT to test cpp and I do not need to support Windows at the moment, just Linux. So with this limitation what would be the correct solution?
Still what is the proper way to set properties (e.g. environment variables to pass to test .cpp etc. ) if test are discovered with gtest_discover_tests?

6

If all you need to is to set MYCONFIG_PATH to a value that will never contain a semicolon, and that’s the only environment variable you need to set, then specifying it with the PROPERTIES keyword in the call to gtest_discover_tests() like in your original post would be the simplest method.

7

That’s what I did in the “PROPERTIES ENVIRONMENT” line in my sample code above but the question I have is about syntax as I tried to use it per documentation “[PROPERTIES name1 value1…” first and itr did not work and example I referred uses “PROPERTIES name1 value1” like in my sample above MYCONFIG_PATH=${CMAKE_CURRENT_SOURCE_DIR}

8

The above should work. It should set the ENVIRONMENT property of each discovered test such that the tests see MYCONFIG_PATH set to ${CMAKE_CURRENT_SOURCE_DIR}. If that isn’t happening for you, please open an issue in the CMake issue tracker with a minimal project that reproduces the problem.