TLDR: I do not understand the behavior of add_custom_command/add_custom_target when multiple of add_custom_command/add_custom_target depends on the same generated file created by add_custom_command in the same directory. It is causing commands unnessariliy run multiple times and may cause random build error when build is done in parallel, and CMake gives no warning when this occurs. Is this an intended behavior?
The current CMake document describes the “DEPENDS” keyword of add_custom_command as the following
If any dependency is an OUTPUT of another custom command in the same directory (CMakeLists.txt file), CMake automatically brings the other custom command into the target in which this command is built.
Consider the following CMakeFiles.txt example. I am using CMake 3.19.1 and CentOS 7.3. Old CMake version has the same issue.
add_custom_command(
OUTPUT output.txt
DEPENDS input.txt
COMMAND touch output.txt
COMMENT “Generating output.txt”)
foreach (i RANGE 100)
add_custom_target(job${i} ALL DEPENDS output.txt)
endforeach (i)
Run this CMakeLists.txt by
cmake . && touch input.txt && make -j
So I created 100 targets. All of them depends on output.txt.
I would like to generate output.txt only once first, then run all 100 targets in parallel.
However, the output contains lots of “Generating output.txt”, implying the output.txt
is generated multiple times in parallel. In this simple CMakeLists.txt, no build error would occur. But if output is generated by a complex command, build could fail randomly, just because two identical commands running in parallel and writing into the same path. This behavior is confusing and CMake gives no warning about this.
I know the issue can be workaround by ensuring generated file is only directly used by
add_custom_target/add_custom_command
add_custom_command(
OUTPUT output.txt
DEPENDS input.txt
COMMAND touch output.txt
COMMENT “Generating output.txt”)
add_custom_target(gen_output DEPENDS output.txt)
foreach (i RANGE 100)
add_custom_target(job${i} ALL DEPENDS gen_output)
endforeach (i)
But I wonder is this the indended usage? I do not want to add one target every time
when the output of add_custom_command is the argument to the DEPENDS keyword of add_custom_command/add_custom_target multiple times in the current CMakeLists.txt
Is there a way to workaround this without adding a custom target?
Edit history
- Reformat the post
- Update TLDR
- Add questions at the end