# using Catch2 test detection when test binary is output outside the build tree

**URL:** https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340
**Category:** Usage
**Created:** [January 7, 2025, 9:45am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340 "2025-01-07T09:45:41Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![adaldev](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/a/9dc877/32.png) [@adaldev](https://discourse.cmake.org/u/adaldev)
#### Post date: [January 7, 2025, 9:45am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340/1 "2025-01-07T09:45:41Z")

</div>

Hi,

I’m using the devel branch of Catch2 and can run simple tests using Catch2 directly.

Now I tried to configure CTest for my project, using Catch2 automation:

```cmake
include(CTest)
include(Catch)
catch_discover_tests(${TARGET_NAME})

```

On build, I’ve got:

> CatchAddTests.cmake:138 (string):  
> string sub-command JSON failed parsing json string: \* Line 1, Column 1  
> Syntax error: value, object or array expected.

In Catch2 utility, it corresponds to:

```cmake
  # Parse JSON output for list of tests/class names/tags
  string(JSON version GET "${listing_output}" "version")
  if (NOT version STREQUAL "1")
    message(FATAL_ERROR "Unsupported catch output version: '${version}'")
  endif()

```

Regards,  
A.

PS strangely enough, the build passed once but running CTest from the build dir obviously returned:

> Unable to find executable: \<Target name\>\_UT\_NOT\_BUILT-\<Some code\>

---

<div class="post-metadata">

### Author: ![Angew](https://discourse.cmake.org/user_avatar/discourse.cmake.org/angew/32/229_2.png) [@Angew](https://discourse.cmake.org/u/Angew)
#### Post date: [January 7, 2025, 10:19am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340/2 "2025-01-07T10:19:07Z")

</div>

Have you tried inspecting the value of `listing_output` to see what is in there and why there might be problems parsing it?

---

<div class="post-metadata">

### Author: ![Angew](https://discourse.cmake.org/user_avatar/discourse.cmake.org/angew/32/229_2.png) [@Angew](https://discourse.cmake.org/u/Angew)
#### Post date: [January 7, 2025, 10:27am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340/4 "2025-01-07T10:27:10Z")

</div>

It’s referenced in the “Catch2 utility” section of your very post. I assume that comes from the `CatchAddTests.cmake` file. You should be able to insert a suitable inspection command in there.

---

<div class="post-metadata">

### Author: ![adaldev](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/a/9dc877/32.png) [@adaldev](https://discourse.cmake.org/u/adaldev)
#### Post date: [January 7, 2025, 10:36am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340/5 "2025-01-07T10:36:31Z")

</div>

I Added a `message` command within `Catch2` `cmake` function. Sometime the build passes (and I’ve got the other error mentioned) but I can’t see the message.  
But it lead me to observe that when running `cmake --build .` twice “works”.  
The first time I’ve got the error, the second time it’s OK.

Yet, even when the build passes, I still got the `NOT_BUILD` issue.

UPDATE:  
I finally managed to get the display.  
It seems that the `Catch2` utilities try to run some special version of the test to do it’s job that output the expected json string. Yet, during compilation the test also compiles one of my files that does some debug output before `main()` (during initialisation of static variable), leading to an ill-formed string.  
Yet, on the second pass, it does not seem to look for the json string anymore.  
There’s something I don’t get in `Catch2` discovery process…

UPDATE2:  
from the documentation

> This function \<catch\_discover\_tests\> works by running the resulting executable with --list-test flag, and then parsing the output to find all existing tests.

Thus it knows where to look for the executable! But it does not pass the info to `CTest` files.  
Could it be a `Catch2` bug? So far I still consider it as a misusage on my side.

---

<div class="post-metadata">

### Author: ![TJNeos](https://discourse.cmake.org/user_avatar/discourse.cmake.org/tjneos/32/5362_2.png) [@TJNeos](https://discourse.cmake.org/u/TJNeos)
#### Post date: [February 22, 2025, 2:54am UTC](https://discourse.cmake.org/t/using-catch2-test-detection-when-test-binary-is-output-outside-the-build-tree/13340/6 "2025-02-22T02:54:35Z")

</div>

> Yet, during compilation the test also compiles one of my files that does some debug output before `main()`

The key is `debug output`.

I had this same issue when I recently updated Catch2. Catch2 appears to now capture anything printed to stdout, and merges that into the in-memory JSON it uses for test discovery. I disabled any debug output before `Catch::Session().run(argc, argv);` which fixed the error messages and cmake was happy again.