`CTest --preset` doesn't appear to allow for out-of-tree builds

When I configure a build in CMake using a build-preset, I can have a handy default for the build dir (e.g. --preset=release can default to build/Release)… but I can always use -B build_dir to override this if I want (e.g., if I need a completely-out-of-tree build directory, for buildbots or other unusual environments). This is great!

Unfortunately, it doesn’t seem possible to use test-presets in CTest with this kind of setup: there doesn’t seem to be a flag to override the default for binaryDir in the test preset (using --test-dir appears to do nothing in this situation).

This seems like a disappointing oversight for presets; ideally, there would be a (common?) flag that could be used to override the binaryDir for a given preset. The only workaround I’ve found is to maintain a separate set of presets, which is something that really should be necessary.

Is there any way to make this work as of recent versions of CMake? If not, could we find a way to make it work in the future?

Example reproducer (courtesy of Alex Reinking):

{
  "version": 3,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 22,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": "default",
      "displayName": "default",
      "description": "default",
      "generator": "Ninja",
      "binaryDir": "${sourceDir}/build"
    }
  ],
  "testPresets": [
    {
      "name": "default",
      "configurePreset": "default"
    }
  ]
}
cmake_minimum_required(VERSION 3.23)
project(test)

enable_testing()

add_test(NAME check COMMAND true)
$ cmake --preset=default -B override
$ ctest --preset=default --test-dir override
Internal ctest changing into directory: /usr/local/google/home/reinking/test/override
Test project /usr/local/google/home/reinking/test/build
No tests were found!!!
1 Like

I guess there isn’t a workaround? Seems hard to believe I’m the only one who has run into this limitation :slight_smile:

I just want to ping this one last time – it seems really surprising to me that this isn’t currently possible. If anyone could at least confirm (or deny) that this is actually impossible, I’d really appreciate it.

Issue created for this in CMake’s issue tracker here: https://gitlab.kitware.com/cmake/cmake/-/issues/23982