# CTest execution much slower than running the executable

**URL:** https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508
**Category:** Usage
**Tags:** os:windows
**Created:** [April 21, 2022, 8:39am UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508 "2022-04-21T08:39:07Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![tooEarlyForThis](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/t/8c91f0/32.png) [@tooEarlyForThis](https://discourse.cmake.org/u/tooEarlyForThis)
#### Post date: [April 21, 2022, 8:39am UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508/1 "2022-04-21T08:39:07Z")

</div>

I’ve got some issues with my tests, when running with CTest. The tests are written using GTest, in particular there is a parametrized test case with about a hundred cases. But the same holds for other test (cases) as well.

When I start the test executable from command-line or IDE (Qt-creator), the test (all cases instantiations executed) takes about 93ms.  
When I start the test using CTest, it takes 2 seconds **each** , so more than 200s in total.

How can that be?

And, even more important: How can I fix that?

Definition is as follows:

```auto
    add_executable(myTest WIN32 testCases.cpp)
    target_compile_options(myTest PRIVATE -Wall -Wextra -Wpedantic -Werror)
    target_link_libraries(myTest PUBLIC gtest_main)
    gtest_discover_tests(myTest)

```

The test itself is similar to this:

```auto
#include <QVariant>

#include <gtest/gtest.h>

struct test_data
{
    QString typeName;
    QVariant input;
};

class MyTestValues: public testing::TestWithParam<test_data>
{
};

TEST_P(MyTestValues, properties)
{
    auto values = GetParam();
    // do some testing
}

INSTANTIATE_TEST_SUITE_P(
    MyTest,
    MyTestValues,
    testing::Values(
        test_data{ QString{ "bool" }, QVariant{ true } },
        test_data{ QString{ "bool" }, QVariant{ false } },
        test_data{ QString{ "char" }, QVariant::fromValue(std::numeric_limits<char>::min()) },
        test_data{ QString{ "char" }, QVariant::fromValue(std::numeric_limits<char>::max()) },
        test_data{ QString{ "char" }, QVariant::fromValue(static_cast<char>(0)) },
        test_data{ QString{ "int" }, QVariant{ 42 } },
        test_data{ QString{ "int" }, QVariant{ std::numeric_limits<int>::min() } },
        test_data{ QString{ "int" }, QVariant{ std::numeric_limits<int>::max() } },
        test_data{ QString{ "long" }, QVariant::fromValue(static_cast<long>(-1)) },
        test_data{ QString{ "long" }, QVariant::fromValue(std::numeric_limits<long>::min()) },
        test_data{ QString{ "long" }, QVariant::fromValue(std::numeric_limits<long>::max()) }
// omitting many other values
));

```

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [April 21, 2022, 11:22am UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508/2 "2022-04-21T11:22:34Z")

</div>

IIUC, GTest tests are run one at a time by CTest, so it ends up launching the executable for each individual test. There may be a way to run them as a single test, but then you lose granularity at the CTest level. You can also investigate if there is unnecessary setup in the one-test-at-a-time setup that you can reduce.

---

<div class="post-metadata">

### Author: ![tooEarlyForThis](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/t/8c91f0/32.png) [@tooEarlyForThis](https://discourse.cmake.org/u/tooEarlyForThis)
#### Post date: [April 21, 2022, 12:51pm UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508/3 "2022-04-21T12:51:51Z")

</div>

I expected some overhead from running the tests one by one by CTest. I’m aware, that CTest starts the test executable for each test-case explicitly, if configured with `gtest_discover_tests`.

But the values I have here are ridiculous. One of the single test cases executed with CTest takes **20 times** the time all test cases take if run in one go.

I’ve been working for years with CMake/CTest on Linux and never have never seen such numbers. I searched through the CMake files for the project definition, but there is nothing out of the ordinary, I don’t even use pre-test-scripts.

Running a single one of these tests gives me this output:

```auto
Note: Google Test filter = MyTest/MyTestValues.properties/0
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from MyTest/MyTestValues
[RUN] MyTest/MyTestValues.properties/0
[OK] MyTest/MyTestValues.properties/0 (0 ms)
[----------] 1 test from MyTest/MyTestValues (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[PASSED] 1 test.

```

It cannot even record the time it took to execute that test.  
CTest gives me this output:

```auto
29: Test timeout computed to be: 1500
29: Running main() from C:/path/to/build/directory/_deps/googletest-src/googletest/src/gtest_main.cc
29: Note: Google Test filter = MyTest/MyTestValues.properties/0
29: [==========] Running 1 test from 1 test suite.
29: [----------] Global test environment set-up.
29: [----------] 1 test from MyTest/MyTestValues
29: [RUN] MyTest/MyTestValues.properties/0
29: [OK] MyTest/MyTestValues.properties/0 (0 ms)
29: [----------] 1 test from MyTest/MyTestValues (0 ms total)
29:
29: [----------] Global test environment tear-down
29: [==========] 1 test from 1 test suite ran. (0 ms total)
29: [PASSED] 1 test.
  1/124 Test #29: MyTest/MyTestValues.properties/24-byteobject<80-A88F-0200-0000-0001-2500-0000-0000-0001-0000-0000-0000-00> ... Passed 1.76 sec

```

GTest still tells me, that the test execution took almost no time, so where do the 1.76 sec CTest takes to execute it come from?

---

<div class="post-metadata">

### Author: ![fenrir](https://discourse.cmake.org/user_avatar/discourse.cmake.org/fenrir/32/734_2.png) [@fenrir](https://discourse.cmake.org/u/fenrir)
#### Post date: [April 21, 2022, 1:48pm UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508/4 "2022-04-21T13:48:36Z")

</div>

It comes mostly from the overhead of starting a new process on Windows (yes, it’s that bad…).

---

<div class="post-metadata">

### Author: ![tooEarlyForThis](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/t/8c91f0/32.png) [@tooEarlyForThis](https://discourse.cmake.org/u/tooEarlyForThis)
#### Post date: [April 21, 2022, 2:03pm UTC](https://discourse.cmake.org/t/ctest-execution-much-slower-than-running-the-executable/5508/5 "2022-04-21T14:03:53Z")

</div>

😭

I already regret taking on that new job