# How to check for incorrect dependencies when using add\_custom\_command?

**URL:** <https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267>\
**Category:** Code\
**Created:** [October 24, 2023, 4:40pm UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267 "2023-10-24T16:40:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Griffon26](https://discourse.cmake.org/user_avatar/discourse.cmake.org/griffon26/32/3946_2.png) [@Griffon26](https://discourse.cmake.org/u/Griffon26)\
**Post date:** [October 24, 2023, 4:40pm UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267/1 "2023-10-24T16:40:59Z")

</div>

Recently I have discovered that I have a problem in my CMake file that can lead to a generator for source files being run while the sources are being compiled. I think I know how to fix this instance, but what I am looking for is a way to discover if any other dependency problems are present in my makefiles.

I could try to build many times and vary how many parallel build processes are running, but I feel that is unlikely to discover these problems. Given that with these makefiles certain source files can be generated while they are also being read would suggest that there is a smarter check that can be done (on a dependency tree perhaps?).

Here’s the code with the problem:

```auto
execute_process(
    COMMAND ${MY_GENERATOR_EXE} --input ${GENERATOR_INPUT_DIR}/main_generator_input.txt --output ${GENERATOR_OUTPUT_DIR} --listfiles
    OUTPUT_VARIABLE GENERATED_SOURCE_FILES
)

add_custom_command(
    OUTPUT ${GENERATED_SOURCE_FILES}
    DEPENDS ${GENERATOR_INPUT_DIR}/*.*
    MAIN_DEPENDENCY ${GENERATOR_INPUT_DIR}/main_generator_input.txt
    COMMAND ${MY_GENERATOR_EXE} --input ${GENERATOR_INPUT_DIR}/main_generator_input.txt --output ${GENERATOR_OUTPUT_DIR} 
)
 
add_custom_target(generator_target
    DEPENDS ${GENERATED_SOURCE_FILES}
    SOURCES ${GENERATOR_INPUT_DIR}/secondary_generator_input1.txt
            ${GENERATOR_INPUT_DIR}/secondary_generator_input2.txt
            ${GENERATOR_INPUT_DIR}/main_generator_input.txt
)

add_library(MyLibrary
    ${GENERATED_SOURCE_FILES}
)

```

I think the fix is adding this:

```auto
set_source_files_properties(${GENERATED_SOURCE_FILES} PROPERTIES GENERATED 1)
add_dependencies(MyLibrary generator_target)

```

Does anyone have ideas on how to look for problems like this?

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [October 29, 2023, 4:59pm UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267/2 "2023-10-29T16:59:24Z")

</div>

> [@Griffon26](#):
>
> I think I know how to fix this instance, but what I am looking for is a way to discover if any other dependency problems are present in my makefiles.

The most fool-proof way is to, for each output (object compile, target, whatever), do:

```commands
$ make clean
$ make -j1 $output

```

If any output is missing dependencies, this will show which ones. Yes, it is a _long_ loop, but you should be able to test a few special instances and assume “others like it” are OK (e.g., if one compilation in a target works and needs outputs generated, others needing a subset of that same output set is also OK).

On to your example:

> [@Griffon26](#):
>
> `DEPENDS ${GENERATOR_INPUT_DIR}/*.*`

This is meaningless; globs are not something CMake models here.

> [@Griffon26](#):
>
> ```auto
> SOURCES ${GENERATOR_INPUT_DIR}/secondary_generator_input1.txt
> ${GENERATOR_INPUT_DIR}/secondary_generator_input2.txt
> 
> ```

~~Nothing seems to declare that it makes these files, so that seems like something is missing.~~ They’re probably in the source tree; no need here. Other custom commands might want to declare these as dependencies though?

> [@Griffon26](#):
>
> ```auto
> execute_process(
> COMMAND ${MY_GENERATOR_EXE} --input ${GENERATOR_INPUT_DIR}/main_generator_input.txt --output ${GENERATOR_OUTPUT_DIR} --listfiles
> OUTPUT_VARIABLE GENERATED_SOURCE_FILES
> )
> 
> ```

Note that you should declare the input file(s) that matter here as `CONFIGURE_DEPENDS` as if the file changes in such a way that the set of output files changes, CMake needs to regenerate the `DEPENDS`.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [October 29, 2023, 9:10pm UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267/3 "2023-10-29T21:10:25Z")

</div>

If you’re using Ninja, you could also try its `missingdeps` tool: [The Ninja build system](https://ninja-build.org/manual.html#_extra_tools)

---

<div class="post-metadata">

**Author:** ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)\
**Post date:** [October 30, 2023, 2:44am UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267/4 "2023-10-30T02:44:42Z")

</div>

That’s a nifty tool, but seems to require `add_custom_command(DEPFILE)` to be used where possible.

---

<div class="post-metadata">

**Author:** ![Griffon26](https://discourse.cmake.org/user_avatar/discourse.cmake.org/griffon26/32/3946_2.png) [@Griffon26](https://discourse.cmake.org/u/Griffon26)\
**Post date:** [October 31, 2023, 2:41pm UTC](https://discourse.cmake.org/t/how-to-check-for-incorrect-dependencies-when-using-add-custom-command/9267/5 "2023-10-31T14:41:43Z")

</div>

Thank you for your feedback. I tried to incorporate it in this updated version.

- I no longer specify a glob in the DEPENDS of add\_custom\_command. Instead I add a file(GLOB) command to create a GENERATOR\_INPUT\_FILES variable. This also allows me to specify CONFIGURE\_DEPENDS as suggested.
- I also added a dependency on the generator to add\_custom\_command, which was missing in my original post
- finally I now have two libraries using the generated sources as an example that shows the type of problem that I am trying to detect. If I forget to add a dependency on the generator\_target to one of the libraries (see commented-out line) then there will still be a dependency on the generated source file, but it can cause the generator to be spawned twice at the same time in a parallel build. This type of problem I would not find by building each individual target from a clean workspace.

CMake knows that the custom command produces those files and it knows that two targets need those files. Isn’t this a type of mistake that CMake should be able to detect at configure time? Or should perhaps the dependency be implicit so that a user cannot even make this mistake?

```auto
project(CMakeTest)
cmake_minimum_required(VERSION 3.25)

set(MY_GENERATOR_EXE python.exe ${PROJECT_SOURCE_DIR}/generator.py)
set(GENERATOR_INPUT_DIR ${PROJECT_SOURCE_DIR}/generator_input)
set(GENERATOR_OUTPUT_DIR ${PROJECT_SOURCE_DIR}/build/generator_output)

file(GLOB GENERATOR_INPUT_FILES CONFIGURE_DEPENDS ${GENERATOR_INPUT_DIR}/*.txt)

execute_process(
    COMMAND ${MY_GENERATOR_EXE} --input ${GENERATOR_INPUT_DIR}/main_generator_input.txt --output ${GENERATOR_OUTPUT_DIR} --listfiles
    OUTPUT_VARIABLE GENERATED_SOURCE_FILES
    COMMAND_ERROR_IS_FATAL ANY
)

add_custom_command(
    OUTPUT ${GENERATED_SOURCE_FILES}
    DEPENDS ${GENERATOR_INPUT_FILES} ${PROJECT_SOURCE_DIR}/generator.py
    MAIN_DEPENDENCY ${GENERATOR_INPUT_DIR}/main_generator_input.txt
    COMMAND ${MY_GENERATOR_EXE} --input ${GENERATOR_INPUT_DIR}/main_generator_input.txt --output ${GENERATOR_OUTPUT_DIR}
)
 
add_custom_target(generator_target
    DEPENDS ${GENERATED_SOURCE_FILES}
    SOURCES ${GENERATOR_INPUT_FILES}
)

add_library(MyLibrary
    ${GENERATED_SOURCE_FILES}
)
add_dependencies(MyLibrary generator_target)

add_library(MyLibrary2
    ${GENERATED_SOURCE_FILES}
)
#add_dependencies(MyLibrary2 generator_target)

```