# Can I test multiple generators with one build of cmake?
**URL:** https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880
**Category:** Development
**Created:** [May 21, 2024, 10:06pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880 "2024-05-21T22:06:49Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 21, 2024, 10:06pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/1 "2024-05-21T22:06:49Z")
As I work on changes to the CMake codebase, it would speed up my testing a lot if I could build `cmake` with Ninja and then use that executable to run the tests with three different generators. Is that possible?
---
### Author:  [@benthevining](https://discourse.cmake.org/u/benthevining)
#### Post date: [May 21, 2024, 10:40pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/2 "2024-05-21T22:40:28Z")
What do you even mean by that? Once you’re running ctest, the cmake generator used for the build is somewhat irrelevant…
---
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 21, 2024, 11:01pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/3 "2024-05-21T23:01:15Z")
That’s exactly my point! The only way I know how to test the `Xcode` generator is by building cmake with Xcode, which is s-l-o-o-o-w:
```auto
cmake -DBUILD_TESTING=1 -S . -B build-xcode -G 'Xcode'
cmake --build build-xcode --config Release
ctest -C Release --test-dir build-xcode --parallel
```
The generator used for testing is automatically chosen to be the generator used for building. Is there a way to override that? I don’t know of one.
Should I know that? Is there a document somewhere that talks about how to test CMake? I’m having other struggles a document like that could answer, like [not knowing](https://discourse.cmake.org/t/expected-failures-on-master/10876/6) how to set up an appropriate environment.
---
### Author:  [@benthevining](https://discourse.cmake.org/u/benthevining)
#### Post date: [May 21, 2024, 11:08pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/4 "2024-05-21T23:08:44Z")
If you want to test binaries built with Xcode, you need to run a build with Xcode, there’s no way around that.
> [@dabrahams](#):
>
> The generator used for testing is automatically chosen to be the generator used for building.
Even if it were possible to build with Ninja and then drive testing with an Xcode project, what exactly would that accomplish? Unless you rebuild with the Xcode toolchain, you’d still be testing binaries built with the Ninja toolchain.
---
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 21, 2024, 11:42pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/5 "2024-05-21T23:42:05Z")
Wow, one of us is **really** not hearing the other, but I can’t tell which of us is failing. Maybe it’s both.
If it’s you, maybe you missed the context that we’re in the “Development” category. I have always assumed this category was about developing CMake itself.
> If you want to test binaries built with Xcode, you need to run a build with Xcode, there’s no way around that.
I don’t. I want to test a `cmake` binary built with Ninja.
_Many_ (most/all?) of CMake’s tests run `cmake` itself to generate a build system for some example project. All of those tests, at least by default, use the same generator to generate that build system. The generator used to generate that build system, at least by default, is the same one I used to build the `cmake` executable under test.
But that `cmake` executable contains code for all of the different generators. I could have broken any of those generators with my changes. It doesn’t matter what generator I used to build the `cmake` executable; in principle I ought to be able to use that `cmake` binary to validate each of the generators for which I have the corresponding build tool.
So I want to build one `cmake` binary with Ninja and then use that binary to validate that I haven’t broken CMake’s `Ninja`, `Xcode`, or `Ninja Multi-Config` generators.
---
### Author:  [@benthevining](https://discourse.cmake.org/u/benthevining)
#### Post date: [May 21, 2024, 11:47pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/6 "2024-05-21T23:47:50Z")
I see. I missed from the original post that you were working on cmake itself, I thought this was a general question about testing any arbitrary cmake project.
---
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 22, 2024, 12:10am UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/7 "2024-05-22T00:10:12Z")
Sorry to have led you astray. I made some changes to the question that I hope will avoid that misunderstanding for others.
---
### Author:  [@fdk17](https://discourse.cmake.org/u/fdk17)
#### Post date: [May 22, 2024, 1:00am UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/8 "2024-05-22T01:00:26Z")
Yes, it should be possible. `CMake_TEST_EXTERNAL_CMAKE` can be used to test an already built `cmake` binary. [https://gitlab.kitware.com/cmake/cmake/-/issues/19785#note\_634196](https://gitlab.kitware.com/cmake/cmake/-/issues/19785#note_634196)
> [@dabrahams](#):
>
> ```auto
> cmake -DBUILD_TESTING=1 -S . -B build-xcode -G 'Xcode'
> cmake --build build-xcode --config Release
> ctest -C Release --test-dir build-xcode --parallel
>
> ```
I’m going by memory; I recall it was something like…
```auto
#build CMake using Xcode
cmake -DBUILD_TESTING=1 -S . -B build-xcode -G 'Xcode'
cmake --build build-xcode --config Release
#test Xcode
ctest -C Release --test-dir build-xcode --parallel
#test Ninja
cmake -DBUILD_TESTING=1 -S . -B test-ninja -G 'Ninja' -DCMake_TEST_EXTERNAL_CMAKE=./build-xcode/bin/Release
ctest --test-dir test-ninja --parallel
```
---
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 22, 2024, 6:31pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/9 "2024-05-22T18:31:33Z")
A slight correction here: `CMake_TEST_EXTERNAL_CMAKE` needs to be an absolute path or it fails, claiming it can’t find `ctest`.
So, on \*nix, the 2nd to last line must be:
```auto
cmake -DBUILD_TESTING=1 -S . -B test-ninja -G 'Ninja' -DCMake_TEST_EXTERNAL_CMAKE=$(pwd)/build-xcode/bin/Release
```
---
### Author:  [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 10, 2024, 12:39pm UTC](https://discourse.cmake.org/t/can-i-test-multiple-generators-with-one-build-of-cmake/10880/10 "2024-06-10T12:39:24Z")
You can also configure with the CMake to test and use `CMake_TEST_HOST_CMAKE=1`.