# Pattern for running tests in-tree and with installed targets?

**URL:** <https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113>\
**Category:** Code\
**Created:** [April 10, 2021, 6:34am UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113 "2021-04-10T06:34:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![morrisonlevi](https://discourse.cmake.org/user_avatar/discourse.cmake.org/morrisonlevi/32/1354_2.png) [@morrisonlevi](https://discourse.cmake.org/u/morrisonlevi)\
**Post date:** [April 10, 2021, 6:34am UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/1 "2021-04-10T06:34:42Z")

</div>

I want to make sure that the targets I export and install are set up correctly. Running the test suite against both the in-tree build and installed version would solve that nicely. Is there a known pattern for this? Something similar to autotool’s distcheck?

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [April 12, 2021, 11:44am UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/2 "2021-04-12T11:44:35Z")

</div>

Nothing exists right now. The project needs to be set up to consume an external project and then build/run its test suite. CMake does it for tests that use the binaries via its `CMake_TEST_HOST_CMAKE` variable. VTK has its [`Testing/External`](https://gitlab.kitware.com/vtk/vtk/-/tree/master/Testing/External) directory that finds VTK then builds and runs the tests against it.

I thought there was an issue for this, but I can’t find it right now. @robert.maynard Do you have that issue link handy?

---

<div class="post-metadata">

**Author:** ![robert.maynard](https://discourse.cmake.org/user_avatar/discourse.cmake.org/robert.maynard/32/4_2.png) [@robert.maynard](https://discourse.cmake.org/u/robert.maynard)\
**Post date:** [April 19, 2021, 6:19pm UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/3 "2021-04-19T18:19:25Z")

</div>

I don’t think I ever filled a formal bug / feature request for this. I only had a draft up on my whiteboard.

---

<div class="post-metadata">

**Author:** ![scivision](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/s/a87d85/32.png) [@scivision](https://discourse.cmake.org/u/scivision)\
**Post date:** [April 20, 2021, 12:31am UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/4 "2021-04-20T00:31:08Z")

</div>

I think I’m doing something similar (using a second, separate “examples” project within the same repo) here and in other projects the same way. I have the CI build and install, then have a separate Examples/ directory with a project that either uses the main project’s build/ directory exports or installed project:

[h5fortran/CMakeLists.txt at main · geospace-code/h5fortran (github.com)](https://github.com/geospace-code/h5fortran/blob/main/Examples/CMakeLists.txt)  
[h5fortran/ci.yml at main · geospace-code/h5fortran (github.com)](https://github.com/geospace-code/h5fortran/blob/main/.github/workflows/ci.yml)

That isn’t one “distcheck” like you wanted or the single superproject-like structure Ben suggests, but does test the same aspects of main project export/install. The reason I took this approach was to avoid adding CMake test code to the main project, but I think it’s up to dev/project preferences.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [April 20, 2021, 10:57pm UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/5 "2021-04-20T22:57:54Z")

</div>

As it happens, I’m doing something very similar in Qt6 at the moment as well. Started out implementing building of examples as test cases using `cmake --build-and-test`, but following discussions with developers, it was decided to add them as `ExternalProject` sub-projects in the main build instead. It means you have to exclude them from the default `all` target and you take on the responsibility of doing an `install` before you try to build the examples, or alternatively you `export()` the necessary things into your build tree and point your examples there instead of an install area - the dependencies also need to be set up carefully to ensure everything needed by the examples have been built. If you want to see the gory details, you can follow along the review of the [changes in gerrit](https://codereview.qt-project.org/c/qt/qtbase/+/334565). Just be aware the Qt has some complex cases to handle that probably won’t apply to most projects, so that adds to the complexity you see there.

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [April 21, 2021, 1:00am UTC](https://discourse.cmake.org/t/pattern-for-running-tests-in-tree-and-with-installed-targets/3113/6 "2021-04-21T01:00:11Z")

</div>

> [@craig.scott](#):
>
> the dependencies also need to be set up carefully to ensure everything needed by the examples have been built.

I moved VTK and ParaView’s examples to be run by the test suite. It ensures they’re run at the right time (in a normal workflow) and instead of checking dependencies at configure time, the tests can skip if they don’t find what they need (with a chance of false positives, but that’s far better than trying to keep the configure and example in sync).