As I work on changes to the CMake codebase, it would speed up my testing a lot if I could build cmake with Ninja and then use that executable to run the tests with three different generators. Is that possible?
What do you even mean by that? Once you’re running ctest, the cmake generator used for the build is somewhat irrelevant…
That’s exactly my point! The only way I know how to test the Xcode generator is by building cmake with Xcode, which is s-l-o-o-o-w:
cmake -DBUILD_TESTING=1 -S . -B build-xcode -G 'Xcode'
cmake --build build-xcode --config Release
ctest -C Release --test-dir build-xcode --parallel
The generator used for testing is automatically chosen to be the generator used for building. Is there a way to override that? I don’t know of one.
Should I know that? Is there a document somewhere that talks about how to test CMake? I’m having other struggles a document like that could answer, like not knowing how to set up an appropriate environment.
If you want to test binaries built with Xcode, you need to run a build with Xcode, there’s no way around that.
Even if it were possible to build with Ninja and then drive testing with an Xcode project, what exactly would that accomplish? Unless you rebuild with the Xcode toolchain, you’d still be testing binaries built with the Ninja toolchain.
Wow, one of us is really not hearing the other, but I can’t tell which of us is failing. Maybe it’s both.
If it’s you, maybe you missed the context that we’re in the “Development” category. I have always assumed this category was about developing CMake itself.
If you want to test binaries built with Xcode, you need to run a build with Xcode, there’s no way around that.
I don’t. I want to test a cmake binary built with Ninja.
Many (most/all?) of CMake’s tests run cmake itself to generate a build system for some example project. All of those tests, at least by default, use the same generator to generate that build system. The generator used to generate that build system, at least by default, is the same one I used to build the cmake executable under test.
But that cmake executable contains code for all of the different generators. I could have broken any of those generators with my changes. It doesn’t matter what generator I used to build the cmake executable; in principle I ought to be able to use that cmake binary to validate each of the generators for which I have the corresponding build tool.
So I want to build one cmake binary with Ninja and then use that binary to validate that I haven’t broken CMake’s Ninja, Xcode, or Ninja Multi-Config generators.
I see. I missed from the original post that you were working on cmake itself, I thought this was a general question about testing any arbitrary cmake project.
Sorry to have led you astray. I made some changes to the question that I hope will avoid that misunderstanding for others.
1 Like
Yes, it should be possible. CMake_TEST_EXTERNAL_CMAKE can be used to test an already built cmake binary. https://gitlab.kitware.com/cmake/cmake/-/issues/19785#note_634196
I’m going by memory; I recall it was something like…
#build CMake using Xcode
cmake -DBUILD_TESTING=1 -S . -B build-xcode -G 'Xcode'
cmake --build build-xcode --config Release
#test Xcode
ctest -C Release --test-dir build-xcode --parallel
#test Ninja
cmake -DBUILD_TESTING=1 -S . -B test-ninja -G 'Ninja' -DCMake_TEST_EXTERNAL_CMAKE=./build-xcode/bin/Release
ctest --test-dir test-ninja --parallel
1 Like
A slight correction here: CMake_TEST_EXTERNAL_CMAKE needs to be an absolute path or it fails, claiming it can’t find ctest.
So, on *nix, the 2nd to last line must be:
cmake -DBUILD_TESTING=1 -S . -B test-ninja -G 'Ninja' -DCMake_TEST_EXTERNAL_CMAKE=$(pwd)/build-xcode/bin/Release
You can also configure with the CMake to test and use CMake_TEST_HOST_CMAKE=1.
1 Like