It will also be easier to work on as a script
That’s a reasonable approach if it’s just unix-land. But “a script” will be at least two separate scripts if you are also having to support windows-land. And I’m not sure why a script will make it work when spawning cmd with args doesn’t, although I’m not sure why that doesn’t work atm either, so who knows. CTest already provides a nice platform-independent way to manage spawning things as a built-in primitive, it feels reasonable to extend that and continue to conceal the platform details in the same place.
The functionality we are talking about here requires adding at least two test cases, one to launch the server as a setup for the fixture and another test case to kill it as a cleanup for the fixture.
Right… CMake already understands there are “fixture” actions external to the foreground test flow that must be orchestrated as properties of the tests; it understands these actions persist for the duration of tests that depend on them since it has the concept of undoing them after the tests that depend on the fixture complete. The official, built-in way to code it today without backgrounding looks like what you describe already. Backgrounding stuff as a property / crossplatform primitive seems to be completely consonant with CTest spawning things crossplatform already and having the “persistent setup and teardown” concept expressed inside an overload of add_test() meaning as today.
A function provided by a module could take care of creating the two test cases so that project code only had to call one command to have both test cases created.
Well, OK, but it’s basically a macro on top of what would have to be able to be done by hand elaborated out. It doesn’t help guide us how it should be done underneath.
check the output of the server
Right… I think that approach that both the background fixture and the foreground code are under test is really the correct way to look at it… often they’re going to be built from the same codebase that’s under test and what we want to test is different bits of the codebase interacting at runtime. Then it seems to lead to the idea that the crossplatform spawning / monitoring / reaping / io capture services already provided by CTest ought to have some parity between provision for foreground test and background fixture, since they are quite profoundly all part of a multi-process test action.
It’s certainly going to be easy to explain to someone familiar with the existing fixture arrangements that there’s just another property and everything else the same to spawn one or more fixture daemons for the test to interact with, and CTest takes care of the crossplatform implications and lifecycle. There is no need for any modules or macros or whatever in that case.
It sounds like we both meet the same situation testwise and feel CTest should handle it, but just differ on whether background fixtures are enough of a primitive that it should actually go in CTest.