# Cannot set property of a test in a subdirectory

**URL:** https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337
**Category:** Code
**Created:** [October 25, 2021, 12:26pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337 "2021-10-25T12:26:38Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![pmqs](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/p/85f322/32.png) [@pmqs](https://discourse.cmake.org/u/pmqs)
#### Post date: [October 25, 2021, 12:26pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/1 "2021-10-25T12:26:38Z")

</div>

I’m having trouble setting properties on all tests in a directory tree.

Here is a dummy project that illustrates the issue

```auto
cmake_minimum_required(VERSION 3.17)
project(Hello_world VERSION 1.0)
enable_testing()

# get_all_tests will walk from CMAKE_CURRENT_SOURCE_DIR & find all tests
function(get_all_tests _out_var)
    set(${_out_var} "")
    get_all_tests_walker(${_out_var} ${CMAKE_CURRENT_SOURCE_DIR})

    set(${_out_var} "${${_out_var}}" PARENT_SCOPE)

endfunction()

function(get_all_tests_walker _out_var _directory)

    get_property(_subdirs DIRECTORY "${_directory}" PROPERTY SUBDIRECTORIES)

    foreach(_subdir IN LISTS _subdirs)
        get_all_tests_walker(${_out_var} "${_subdir}")
    endforeach()

    get_directory_property(_tests DIRECTORY "${_directory}" TESTS)
    set(${_out_var} ${${_out_var}} ${_tests} PARENT_SCOPE)

endfunction()

# add a subdir with a test
add_subdirectory(another_dir)

# add a test local to the current directory

add_test(
    NAME localtest
    COMMAND localtest
)

# read all test names into "alltests"
get_all_tests(alltests)

message(WARNING "${alltests}")

set_property(TEST ${alltests}
        APPEND
        PROPERTY
            LABELS PROTOBUF mylabel
)

```

The directory **another-dir** has this

```auto
add_test(
    NAME subdirtest
    COMMAND mycommand
)

```

When I run cmake against the above I get this output

```auto
$ cmake ..
CMake Warning at CMakeLists.txt:126 (message):
  subdirtest;localtest

CMake Error at CMakeLists.txt:141 (set_property):
  set_property given TEST names that do not exist:

    subdirtest

-- Configuring incomplete, errors occurred!

```

**Observations**

1. the call to `get_all_tests` has correctly found all the tests in the directory tree

2. the call to `set_property` cannot find the **subdirtest** test  
It appears that `set_property(TEST` only sees tests in the current directory

Is this defined behaviour or have I missed something obvious?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [October 26, 2021, 9:00pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/2 "2021-10-26T21:00:36Z")

</div>

It does indeed seem that setting a test property only looks for the test in the current directory scope. The docs should be improved to mention this.

---

<div class="post-metadata">

### Author: ![pmqs](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/p/85f322/32.png) [@pmqs](https://discourse.cmake.org/u/pmqs)
#### Post date: [October 26, 2021, 9:34pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/3 "2021-10-26T21:34:55Z")

</div>

Is the current behaviour of limiting the scope to the current directory the intended behaviour or is it a bug/oversight in the current implementation?

If this is defined behaviour, can I raise enhancement request?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [October 26, 2021, 10:28pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/4 "2021-10-26T22:28:14Z")

</div>

It seems to be intended. Modifying tests in other directories is certainly _possible_, but it doesn’t look trivial (the directory scope tree would need walked to find the test, though whatever mechanism guarantees test name uniqueness could be reused (and enhanced if necessary) to do a more direct lookup.

Sure, file an issue. No guarantees on any timeline to actually changing it (though MRs are certainly welcome).

---

<div class="post-metadata">

### Author: ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)
#### Post date: [October 27, 2021, 9:05pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/5 "2021-10-27T21:05:27Z")

</div>

As far as I’m aware, test names are not guaranteed nor required to be unique beyond the directory scope in which they are defined, possibly not even within a directory scope. Even CMake’s own test suite uses duplicated test names in some places. (sorry, had to edit my original statement after realising it said almost the opposite of what I intended)

---

<div class="post-metadata">

### Author: ![pmqs](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/p/85f322/32.png) [@pmqs](https://discourse.cmake.org/u/pmqs)
#### Post date: [October 27, 2021, 10:02pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/6 "2021-10-27T22:02:17Z")

</div>

> [@craig.scott](#):
>
> Test names are not guaranteed nor required to be unique, at least not within the same directory scope.

Yep - just tried that out - within a subdirectory all test names need to be unique, but once you span multiple directories you can have duplicates. Damn!

That then suggests that the technique of walking a directory tree to collect all the tests using the existing implementation of `get_directory_property(...TESTS)` is never going to work, even if `set_property(TEST` was somehow enhanced to work across directories. You can never guarantee that the list of tests is complete if there are potential duplicates.

Are there any other techniques available that would allow test properties to be updated that don’t just involve making the change at the point where the call to `add_test` is made, like below?

```auto
add_test(
    NAME localtest
    COMMAND some command
)

set_property(
    TEST
        localtest
    APPEND PROPERTY
        ENVIRONMENT FRED=${FRED}
)

```

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [October 28, 2021, 12:31am UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/7 "2021-10-28T00:31:29Z")

</div>

A function can do it (it has a different variable scope, but inherits the directory scope from the caller). So something like:

```cmake
add_test(NAME blah)
apply_my_test_props(blah)

```

---

<div class="post-metadata">

### Author: ![pmqs](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/p/85f322/32.png) [@pmqs](https://discourse.cmake.org/u/pmqs)
#### Post date: [October 28, 2021, 8:31am UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/8 "2021-10-28T08:31:28Z")

</div>

That doesn’t help because I need to span directory scopes.

Let me rephrase my question – are there any other techniques available that would allow test properties from a different directory scope to be updated?

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [October 28, 2021, 11:44am UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/9 "2021-10-28T11:44:25Z")

</div>

Not that I’m aware of, no.

---

<div class="post-metadata">

### Author: ![pmqs](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/p/85f322/32.png) [@pmqs](https://discourse.cmake.org/u/pmqs)
#### Post date: [October 28, 2021, 1:18pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/10 "2021-10-28T13:18:43Z")

</div>

OK. I’ve created issue [https://gitlab.kitware.com/cmake/cmake/-/issues/22813](https://gitlab.kitware.com/cmake/cmake/-/issues/22813) to record the issue anyway

---

<div class="post-metadata">

### Author: ![starball](https://discourse.cmake.org/user_avatar/discourse.cmake.org/starball/32/3437_2.png) [@starball](https://discourse.cmake.org/u/starball)
#### Post date: [May 17, 2023, 5:13pm UTC](https://discourse.cmake.org/t/cannot-set-property-of-a-test-in-a-subdirectory/4337/11 "2023-05-17T17:13:09Z")

</div>

Related on Stack Overflow: [Is it possible in CMake to set properties of tests in a different directory scope than the one the test is defined in?](https://stackoverflow.com/q/76270079/11107541)
