# How to test a multi-config generator with multiple configurations
**URL:** https://discourse.cmake.org/t/how-to-test-a-multi-config-generator-with-multiple-configurations/10869
**Category:** Development
**Created:** [May 18, 2024, 1:54am UTC](https://discourse.cmake.org/t/how-to-test-a-multi-config-generator-with-multiple-configurations/10869 "2024-05-18T01:54:23Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@dabrahams](https://discourse.cmake.org/u/dabrahams)
#### Post date: [May 18, 2024, 1:54am UTC](https://discourse.cmake.org/t/how-to-test-a-multi-config-generator-with-multiple-configurations/10869/1 "2024-05-18T01:54:23Z")
I’m trying to modify a test declared with `ADD_TEST_MACRO` to check that multiple configurations can be built with the same build directory and a single `cmake` configure invocation. Is there a non-hacky technique for that? It’s not obvious from the code.
Edit: Actually, let me add a second question here. I found [a hacky way to do it](https://gitlab.kitware.com/dabrahams/cmake/-/commit/edb8ede69e1cbaa0f67ffbc0e27439141d09e8c5#3bd5bb36485bba08bd88da8cc6a94a6900244c90), revealing [a bug in the Ninja Multi-Config generator](https://gitlab.kitware.com/cmake/cmake/-/issues/25997). The nature of the bug is that files that should have been separated per-configuration are not. That seems like an easy mistake to make in all kinds of areas of CMake, not just in Swift-specific code like the test I added. Shouldn’t _all_ the tests be run that way when the generator is multi-config?
Thanks
---
### Author:  [@brad.king](https://discourse.cmake.org/u/brad.king)
#### Post date: [May 21, 2024, 12:56pm UTC](https://discourse.cmake.org/t/how-to-test-a-multi-config-generator-with-multiple-configurations/10869/2 "2024-05-21T12:56:00Z")
When configured with a multi-config generator, running tests with `ctest` requires specifying what configuration to test. For example, `ctest -C Debug` runs tests using the `bin/Debug/{cmake,ctest,...}` binaries from the current build tree. If one has built multiple configurations of CMake itself, the test suite can be run separately in each configuration. The tests themselves sometimes have config-specific behavior, but we have no model for testing multiple configurations simultaneously.
We do have tests for per-config artifact locations, but none for Swift.
---
### Author:  [@brad.king](https://discourse.cmake.org/u/brad.king)
#### Post date: [May 21, 2024, 1:32pm UTC](https://discourse.cmake.org/t/how-to-test-a-multi-config-generator-with-multiple-configurations/10869/3 "2024-05-21T13:32:45Z")
In general when you want more control over the testing sequence, such as configuring once and building with multiple configurations, the “RunCMake” infrastructure is best. See `Tests/RunCMake/README.rst`. Grep through `Tests/RunCMake/*/RunCMakeTest.cmake` for use of `--build` to see cases where we run the build after running cmake in the test.