# Unit tests failing only when run via CTest

**URL:** https://discourse.cmake.org/t/unit-tests-failing-only-when-run-via-ctest/2694
**Category:** Usage
**Created:** [February 4, 2021, 9:21am UTC](https://discourse.cmake.org/t/unit-tests-failing-only-when-run-via-ctest/2694 "2021-02-04T09:21:18Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![devmake](https://discourse.cmake.org/user_avatar/discourse.cmake.org/devmake/32/1178_2.png) [@devmake](https://discourse.cmake.org/u/devmake)
#### Post date: [February 4, 2021, 9:21am UTC](https://discourse.cmake.org/t/unit-tests-failing-only-when-run-via-ctest/2694/1 "2021-02-04T09:21:18Z")

</div>

Have the following situation:

1. C++ codebase
2. Unit test suite based on the Boot.Test
3. When run directly all the tests pass no issues
4. When run via ctest eg: **ctest -j 1 or ctest -j 5** tests fail
5. It is the same test every time
6. Only happens with a specific C++ compiler (clang6) and only in release builds
7. Same codebase and UTs built with other compilers GCC and other versions of clang work fine
8. The UTs don’t use shared memory or environment variables etc.
9. Happens with or without **–force-new-ctest-process**

* * *

**Q1:** Has anyone encountered a situation like this before?

**Q2:** How would one go about investigating whats going on?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [February 4, 2021, 2:51pm UTC](https://discourse.cmake.org/t/unit-tests-failing-only-when-run-via-ctest/2694/2 "2021-02-04T14:51:10Z")

</div>

I would run that test with verbosity (`ctest -R '$testname' -VV`) to see how CTest is invoking it. Things to note:

- CTest may be setting extra environment settings (via the `ENVIRONMENT` property)
- Clang could have a bug (6 is pretty old these days); release-only points to something optimizer-related

For debugging, I would run the test inside and outside of CTest using `valgrind` to see if it finds anything (you’ll need to modify the `add_test` to have it run under CTest). You can also try the sanitizers (ASan and UBSan are of particular note), though Clang 6’s sanitizers may be less useful than new ones (that said, new ones could also point the problem out without crashing too).