Do we have any nightly builds which run cmake and ctest from a directory other than the build directory? If not, we should try to get at least one build site to do that. This might help catch things like situations that apply to policy CMP0070 and issue 20885. I’m wondering how many other similar issues might be lurking as well. Maybe we could also potentially modify some of the test cases to do that too, but getting a good cross-section of feature coverage may be difficult.
Please clarify what you mean by “from a directory other than the build”. Do you mean the working directory, the location of the cmake executable, or something else?
I meant the working directory should be something other than the top of the build directory. The following scenarios come to mind:
- People invoking
cmake -S xxx -B yyyfrom a directory that is neitherxxxnoryyy. This might be more common when defining the steps of a CI job. - Running
ctestin a subdirectory of the build directory. This can be used to run a subset of the tests based on directory structure.
I more had invoking cmake in mind when I wrote the original post, but I added ctest too after realising it may also have cases that could be relevant. My comments about build sites more apply to the cmake case than ctest (I think the latter would be more about test cases than the setup of build sites).
Most of the test suite drives its own runs of ctest and/or cmake so the working directory of the outer ctest wouldn’t matter much. Many of the RunCMake tests could probably be updated to use -S and -B with a separate working directory. Many other tests use ctest --build-and-test, which currently relies on the working directory for invoking its internal cmake too. An option could be added to that to switch to -S and -B for some tests.
I’m increasingly thinking this will be valuable. I just found another case while working on the MR for the file(ARCHIVE_EXTRACT) updates.