I’m building some libraries, all of which generate some of their files.
I’ve created the generated files using add_custom_command(), but I’ve discovered something which seems like a false dependency. If a downstream library has a generated file and links to an upstream library, the downstream library sources will not start to compile until the upstream library is completely built. With many libraries (more than 50) in my project, this false dependency causes serialization in the build.
What’s curious is that I also noticed that if an explicit add_custom_target() for the generated file is used with add_dependencies(), the false dependency no longer exists, and the files in the downstream library will compile concurrently with the ones in the upstream library. So I have a workaround, but is this expected behavior?
Using Cmake 1.19, Ninja 1.10.2. Behavior is identical on macOS and Linux.
The following is a minimal CMakeLists.txt file that shows what happens. The WORKS option conditionally adds the add_custom_target() and add_dependencies() clauses which cause it to work (quickly). The files foo.c and bar.c are empty, and I’m building in a subdirectory of the CMakeLists.txt directory.
I put a file called /tmp/x which is a wrapper around cc that sleeps to show the serialization occur:
#!/bin/bash -e
echo "mycc" $(date)
sleep 4
exec /usr/bin/cc $*
Here is the CMakeLists.txt:
cmake_minimum_required(VERSION 3.19)
project(test)
option(WORKS "false dependency gone if WORKS set to ON" OFF)
add_library(foo
foo.c
)
add_library(bar
bar.c
bargen.h
)
target_include_directories(bar PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
add_custom_command(OUTPUT bargen.h
COMMAND touch bargen.h
)
if (${WORKS})
add_custom_target(generate_file
DEPENDS bargen.h
)
add_dependencies(bar generate_file)
endif()
target_link_libraries(bar PUBLIC foo)
Build it like this and you will see the serialization: bar.c will not start to compile until after foo.c has finished compiling (in fact, not until after the foo library is built). (I’m explicitly choosing ‘-j 4’ to ensure Ninja will try to build in parallel).
cmake -DWORKS=OFF -G Ninja -DCMAKE_C_COMPILER=/tmp/x .. && ninja clean && ninja -j 4 -v | grep mycc
This shows the following output:
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/rhb/Downloads/cmake-anomaly/build
[1/1] Cleaning all built files...
Cleaning... 5 files.
mycc Sat Jan 2 15:15:25 EST 2021
mycc Sat Jan 2 15:15:29 EST 2021
Now build it like this and you’ll see that bar.c compiles concurrently with foo.c:
cmake -DWORKS=ON -G Ninja -DCMAKE_C_COMPILER=/tmp/x .. && ninja clean && ninja -j 4 -v | grep mycc
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/rhb/Downloads/cmake-anomaly/build
[1/1] Cleaning all built files...
Cleaning... 5 files.
mycc Sat Jan 2 15:15:37 EST 2021
mycc Sat Jan 2 15:15:37 EST 2021
.