# Testing in a superbuild

**URL:** <https://discourse.cmake.org/t/testing-in-a-superbuild/1906>\
**Category:** Code\
**Created:** [September 24, 2020, 9:11am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906 "2020-09-24T09:11:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![alex](https://discourse.cmake.org/user_avatar/discourse.cmake.org/alex/32/125_2.png) [@alex](https://discourse.cmake.org/u/alex)\
**Post date:** [September 24, 2020, 9:11am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/1 "2020-09-24T09:11:06Z")

</div>

If I have a superbuild, ie. one that just consists of a sequence of `ExternalProject` calls, how can I use CTest to run _all_ of the tests in _all_ of the subprojects? Using `enable_testing()` won’t do anything, but maybe there’s some way to generate a CTest file that `include()`s the subproject files?

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [September 25, 2020, 4:48am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/2 "2020-09-25T04:48:37Z")

</div>

I’ve done a hack for this in our superbuild infrastructure. You can likely adapt it for your project. The files in question:

- [the logic](https://gitlab.kitware.com/paraview/common-superbuild/-/blob/a843781c0a5d310f9b723eda311e8a789d82be76/CMakeLists.txt#L185-233)
- [the trampoline](https://gitlab.kitware.com/paraview/common-superbuild/-/blob/master/cmake/superbuild_testing_trampoline.cmake.in)

The logic gathers the projects we want to test, extracts their binary directories and configures a `CTestTestfile.cmake` to do the work. It then uses the trampoline to do the actual `subdirs` calls for each project we’re interested in. This file is added to the list of files for CTest to include.

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [September 25, 2020, 4:49am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/3 "2020-09-25T04:49:50Z")

</div>

You actually don’t need the custom `CTestTestfile.cmake` most likely…that is an oddity of the way that project is meant to be used.

---

<div class="post-metadata">

**Author:** ![alex](https://discourse.cmake.org/user_avatar/discourse.cmake.org/alex/32/125_2.png) [@alex](https://discourse.cmake.org/u/alex)\
**Post date:** [November 1, 2021, 4:36am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/4 "2021-11-01T04:36:36Z")

</div>

I just used this today to make a cross-compiling superbuild experience a bit better…

The minimal version of this approach that I’m using is the following:

```cmake
ExternalProject_Get_Property(cross-build BINARY_DIR)
file(RELATIVE_PATH ctest_dir "${CMAKE_CURRENT_BINARY_DIR}" "${BINARY_DIR}")
file(CONFIGURE OUTPUT "CTestTestfile.cmake"
     CONTENT [[
subdirs("${ctest_dir}")
]])

```

and I just take care to not call `enable_testing()` in my (very short) super-build.

---

<div class="post-metadata">

**Author:** ![dbabokin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dbabokin/32/1930_2.png) [@dbabokin](https://discourse.cmake.org/u/dbabokin)\
**Post date:** [November 2, 2021, 7:54pm UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/5 "2021-11-02T19:54:50Z")

</div>

Is there any chance that this functionality can be integrated into ExternalProject\_Add()? That would be very helpful if this is available out of the box rather than through a hack.

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [November 3, 2021, 4:23pm UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/6 "2021-11-03T16:23:13Z")

</div>

Given the horrible hack it does to make `subdirs()` work, I think the proper way is to add support to CTest to chain into other `CTestTestfile.cmake` files given a list of directories rather than…that 🙂 .

---

<div class="post-metadata">

**Author:** ![dbabokin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dbabokin/32/1930_2.png) [@dbabokin](https://discourse.cmake.org/u/dbabokin)\
**Post date:** [November 3, 2021, 5:24pm UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/7 "2021-11-03T17:24:01Z")

</div>

Yeah, the hack with `subdirs()` is a hack. I’m not sure where the proper fix should be. From the user prospective, I would prefer to have an extra parameter for `ExternalProject_Add()` that would signal that tests need to be exported to the parent context. Not sure that it is possible, through. But that would be ideal.

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [November 4, 2021, 11:02am UTC](https://discourse.cmake.org/t/testing-in-a-superbuild/1906/8 "2021-11-04T11:02:41Z")

</div>

I think first would be to add some support to CTest for “chain off into this other directory” natively (probably via a directory property like `CTEST_ADDITIONAL_SUBDIRS` or something). Once that is done, `ExternalProject_add` can gain an argument like `BUILD_ADD_TO_CTEST 1` or something to hook into that property.