Hello all,
I am running into an execution inconsistency between gtest and ctest. Our project is a legacy project that original had a recursive make system but has been updated to a cmake system collecting old tests as well as tests that have been converted to gtests. We are executing all tests through ctest using -R to select specific tests to run.
In converting some of our integration tests, tests that run a series of apps in a pipeline, I was trying to write a gtest suite level fixture, see the docs here. As per the gtest documentation this should run on a per suite basis, where the majority of our suites are defined by a fixture.
This is where ctest and gtest seem to diverge. If my suite is called Foo, and I run ctest -R Foo rather than running all tests under the Foo suite, ctest seems to select each tests individually and run the gtest executable as:
testexe --gtest_filter=“Foo.test1”
Running 1 test from 1 test suite.
…
testexe --gtest_filter=“Foo.test2”
Running 1 test from 1 test suite.
…
testexe --gtest_filter=“Foo.test3”
Running 1 test from 1 test suite.
…
(Other formatting ommited)
Where each test runs the setup and teardown for the test suite.
If I run the executable directly we see:
testexe --gtest_filter=“Foo*”
Running 3 tests from 1 test suites.
(Other formatting omitted)
The suite level fixture will be setup once, and torn down once since all tests are run together as a single suite.
My question is, does ctest retain some functionality with gtest that allows all suites to be run together? Is there some particular command line argument that should be supplied to maintain this behavior?
I haven’t been able to find any particularly good solutions. One such solution being override some ctest functionality through our cmake system, which is something I would like to avoid.
Hopefully that communicates the issues we are running into, and I appreciate all the work that yall do.