ctest 3.16 - tests don't timeout if there are remaining subprocesses

Usage
1

Hi there,

I noticed that ctest 3.16 fails to timeout tests if their main process has terminated, but there are still subprocesses running. (NB : Ubuntu 20.04 ships cmake 3.16.3)

For example, here is a test script that starts a subprocess (in the background), and terminates immediately :

#!/bin/bash
sleep 10&

And here is a CMakeLists.txt that adds a test named repro :

cmake_minimum_required(VERSION 3.12)
project(repro)

include(CTest)
enable_testing()

add_test(NAME repro COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/test.sh)
set_tests_properties(repro PROPERTIES TIMEOUT 5)

Then if we run the test with ctest 3.16, we can see it does not timeout :

$ mkdir build && cd build && cmake ..
$ ctest --verbose
[...]
test 1
    Start 1: repro

1: Test timeout computed to be: 5
1/1 Test #1: repro ............................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =  10.01 sec

Whereas with more recent versions of cmake, it does timeout properly:

$ ctest --verbose
[...]
1: Test timeout computed to be: 5
1/1 Test #1: repro ............................***Timeout   5.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   5.01 sec

The following tests FAILED:
          1 - repro (Timeout)
Errors while running CTest

More precisely,

  • I could reproduce the issue with cmake 3.16.3 (the version shipped with Ubuntu 20.04), and 3.16.9 (the latest patch version)
  • I could not reproduce the issue with cmake 3.17.5, 3.18.3, or 3.19.2
  • I couldn’t find anything seemingly related in the release notes of 3.17

Should I open an issue on gitlab ? (3.16 is kinda old, I don’t know if it is maintained.)

Anyway, I thought I’d share the information.

2

This probably coincides with the kwsys -> libuv process management transition. Not sure what to do about it as it seems that 3.17+'s behavior makes more sense to me.

Cc: @brad.king

3

This was indeed fixed in !4217, which was first present in 3.17.0.

We are not providing any further updates to the 3.16 branch. Since you’re using Ubuntu, I would suggest using the newer builds from our APT repository.

4

(Ha! I check open issues, but not closed ones…)

Thank you very much for your replies. We’ll use a more recent version of cmake.