using Catch2 test detection when test binary is output outside the build tree

Usage
1

Hi,

I’m using the devel branch of Catch2 and can run simple tests using Catch2 directly.

Now I tried to configure CTest for my project, using Catch2 automation:

include(CTest)
include(Catch)
catch_discover_tests(${TARGET_NAME})

On build, I’ve got:

CatchAddTests.cmake:138 (string):
string sub-command JSON failed parsing json string: * Line 1, Column 1
Syntax error: value, object or array expected.

In Catch2 utility, it corresponds to:

  # Parse JSON output for list of tests/class names/tags
  string(JSON version GET "${listing_output}" "version")
  if (NOT version STREQUAL "1")
    message(FATAL_ERROR "Unsupported catch output version: '${version}'")
  endif()

Regards,
A.

PS strangely enough, the build passed once but running CTest from the build dir obviously returned:

Unable to find executable: <Target name>_UT_NOT_BUILT-<Some code>

2

Have you tried inspecting the value of listing_output to see what is in there and why there might be problems parsing it?

4

It’s referenced in the “Catch2 utility” section of your very post. I assume that comes from the CatchAddTests.cmake file. You should be able to insert a suitable inspection command in there.

5

I Added a message command within Catch2 cmake function. Sometime the build passes (and I’ve got the other error mentioned) but I can’t see the message.
But it lead me to observe that when running cmake --build . twice “works”.
The first time I’ve got the error, the second time it’s OK.

Yet, even when the build passes, I still got the NOT_BUILD issue.

UPDATE:
I finally managed to get the display.
It seems that the Catch2 utilities try to run some special version of the test to do it’s job that output the expected json string. Yet, during compilation the test also compiles one of my files that does some debug output before main() (during initialisation of static variable), leading to an ill-formed string.
Yet, on the second pass, it does not seem to look for the json string anymore.
There’s something I don’t get in Catch2 discovery process…

UPDATE2:
from the documentation

This function <catch_discover_tests> works by running the resulting executable with --list-test flag, and then parsing the output to find all existing tests.

Thus it knows where to look for the executable! But it does not pass the info to CTest files.
Could it be a Catch2 bug? So far I still consider it as a misusage on my side.