I’m wondering if we might be able to change the default value of the new
DISCOVERY_MODE option for
gtest_discover_tests(), added in the CMake 3.18 release. We are still in release candidate stage, so it’s not too late to consider it. At the moment, the default is
POST_BUILD, which maps to the existing behaviour. The other valid value is
PRE_TEST, which delays discovery until
ctest actually runs. There are robustness benefits to
PRE_TEST, as explained in the description of the merge request that added this feature.
Ordinarily, we would provide a policy for any change in behaviour, but I’m really wondering if that would be necessary here. My rationale is that the test must be runnable at test time, so we know it should be possible to extract the test list from it then. That means if it worked the old way, it should still work with the new way. The only way you should legitimately be obtaining the list of tests from a
POST_BUILD method would be to ask
ctest to tell you, so
PRE_TEST will also still work for that scenario. Basically, I don’t think we are losing any observable behavior, just gaining more robustness. Seems like making the default more robust would be a good idea.
Also note that the default can be changed globally with the
CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE variable, so there’s an easy way to get the old behavior if someone has a strange corner case (likely an unsupported scenario) and they genuinely need it. I can’t think of such a case, but my point is that even if there is, there’s a trivial way to get the old behavior back.
I also just realised that we don’t mention
CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE in the release notes. Maybe we should mention it there.