express the right dependencies in CMake

Code
, ,
#1

Hi,

in my projects I have a header file that is generated at build-time. It is included from a single .c file. So the chain of dependencies would be as follows:
executable -> file.o -> header generated.

How is it possible to express this dependency?
I surfed the internet and found this post https://stackoverflow.com/questions/47481896/how-to-add-a-dependency-to-an-object-file-in-cmake

The proposed solution is to declare the dependency of the executable also on the generated .h file, but so there is no guarantee that the .h file will be created or updated before the .o, or am I wrong?

I have created a small example that I attach here
foo.tar.bz2 (1.5 KB)

just run build.sh

best regards
Max

evaluating cmake
#2

From a very quick look at your example project, it seems to be doing the right things to get the generated file into the build reliably. Dependency relationships will be created to ensure that the header is generated before trying to compile the source that uses it.

For others coming across this question, the following article may be helpful:

That article generates a .cpp file, but the same approach should work for generating headers (assuming you have at least one .cpp file that includes the generated header).

#3

I’m not convinced. The dependency I would like to express (in “pseudo-Makefile” since I’m more familiar with makefiles than ninjas) is as follows:

bar: version.o
	<< link >>

version.o : version.c version_autogen.h
	<< compile >>

version_autogen.h:
	<< create version_autogen.h >>

I tried to take a look at the generated makefiles. I found:

version_autogen.h:
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --blue --bold --progress-dir=/home/max/Lavori/nova/srcs/repo/da_casa/stampcarrier/FW/torm/build/CMakeFiles --progress-num=$(CMAKE_PROGRESS_1) "Creating version_autogen.h"
	../version_creator.sh 1 2 3 alpha.1 version_autogen.h

CMakeFiles/bar.dir/version.c.o: CMakeFiles/bar.dir/flags.make
CMakeFiles/bar.dir/version.c.o: ../version.c
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green --progress-dir=/home/max/Lavori/nova/srcs/repo/da_casa/stampcarrier/FW/torm/build/CMakeFiles --progress-num=$(CMAKE_PROGRESS_2) "Building C object CMakeFiles/bar.dir/version.c.o"
	/usr/bin/cc $(C_DEFINES) $(C_INCLUDES) $(C_FLAGS) -o CMakeFiles/bar.dir/version.c.o   -c /home/max/Lavori/nova/srcs/repo/da_casa/stampcarrier/FW/torm/version.c

CMakeFiles/bar.dir/version.c.i: cmake_force
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green "Preprocessing C source to CMakeFiles/bar.dir/version.c.i"
	/usr/bin/cc $(C_DEFINES) $(C_INCLUDES) $(C_FLAGS) -E /home/max/Lavori/nova/srcs/repo/da_casa/stampcarrier/FW/torm/version.c > CMakeFiles/bar.dir/version.c.i

CMakeFiles/bar.dir/version.c.s: cmake_force
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green "Compiling C source to assembly CMakeFiles/bar.dir/version.c.s"
	/usr/bin/cc $(C_DEFINES) $(C_INCLUDES) $(C_FLAGS) -S /home/max/Lavori/nova/srcs/repo/da_casa/stampcarrier/FW/torm/version.c -o CMakeFiles/bar.dir/version.c.s

There is no direct dependency linking version.c.o to version_autogen.h.
This means that, for make, version.c.o and version_autogen.h are independent of each other, and the order of execution can be any. In a multi-job build there is no guarantee that the generation of version_autogen.h happens before the compilation of version.c.o.
Please correct me if I am wrong.

I’ve read the article you suggest, but I think the case I’m proposing is not covered by the topics in the article. In fact, having a generated .cpp file, it’s possible to express the true dependency between the executable and the generated .cpp.

best regards
Max

#4

Digging deeper into the other files that make reads, in build/CMakeFiles/bar.dir/build.make, I see the following at the end of that file:

CMakeFiles/bar.dir/depend: version_autogen.h
	cd /Users/craig/Projects/gen_header/build && $(CMAKE_COMMAND) -E cmake_depends "Unix Makefiles" /Users/craig/Projects/gen_header /Users/craig/Projects/gen_header /Users/craig/Projects/gen_header/build /Users/craig/Projects/gen_header/build /Users/craig/Projects/gen_header/build/CMakeFiles/bar.dir/DependInfo.cmake --color=$(COLOR)
.PHONY : CMakeFiles/bar.dir/depend

Earlier near the top of that same file is this:

# Include any dependencies generated for this target.
include CMakeFiles/bar.dir/depend.make

Initially after first running CMake, the file at CMakeFiles/bar.dir/depend.make doesn’t hold anything interesting. If you were to ask to build just version.o at this point, it would indeed fail because the Makefiles don’t set up any rules that make the compilation step depend on regenerating the dependencies first. This is probably a bug in CMake’s Makefile generation (@marc.chevrier you may be interested in this). If you were to build the bar target rather than version.o though, it does seem to invoke the header generation first. After you’ve done a build once, the file at CMakeFiles/bar.dir/depend.make will now hold a rule which enforces the missing constraint:

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.19

CMakeFiles/bar.dir/version.c.o: ../version.c
CMakeFiles/bar.dir/version.c.o: version_autogen.h

So yes, there is a small window of opportunity for specific use cases before the first build at least, but after that it is probably enforced. Maybe @marc.chevrier has more insight on this, he’s been working on the dependency handling lately.

#5

File CMakeFiles/bar.dir/depend.make is populated by command $(CMAKE_COMMAND) -E cmake_depends. And target CMakeFiles/bar.dir/depend is always called before any other target, so depend.make is populated by the implicit dependencies before the effective compilation.
But, a new strategy is now used (dependencies are generated by the compiler itself) which imply that implicit dependencies are not populated during the first compilation. To ensure generated headers are created before effective compilation, even the first one, explicit dependency must be declared. As a result of this explicit dependency, the generated header is declared as a dependency target of CMakeFiles/bar.dir/depend.

For that purpose, adding the generated header to the target consuming it is enough and perfectly reliable:

add_custom_command(OUTPUT version_generated.h COMMAND ...)
add_executable(my_exe version_generated.h source.c...)
#6

Unfortunately that isn’t what I observe. Here’s the CMakeLists.txt file that I was using when I found that the initial dependency isn’t correctly enforced (this is just a slightly simplified version of the one in the tarball attached to the original poster’s question):

cmake_minimum_required(VERSION 3.10)
project(foo)

SET (foo_VERSION_MAJOR 1)
SET (foo_VERSION_MINOR 2)
SET (foo_VERSION_PATCH 3)
SET (foo_VERSION_PRERELEASE "alpha.1")

SET(VER_FILE "version_autogen.h")

# add_custom_command does not create a new target. You have to define targets explicitly
# by add_executable, add_library or add_custom_target in order to make them visible to make
add_custom_command(OUTPUT ${VER_FILE}
    COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/version_creator.sh
                ${foo_VERSION_MAJOR}
                ${foo_VERSION_MINOR}
                ${foo_VERSION_PATCH}
                ${foo_VERSION_PRERELEASE}
                ${VER_FILE}

    # Display the given message before the commands are executed at build time
    COMMENT "Creating ${VER_FILE}"
)

add_executable(bar version.c ${CMAKE_BINARY_DIR}/${VER_FILE})
target_include_directories(bar PRIVATE ${CMAKE_BINARY_DIR})

Note how the header is listed as a source for the bar target, and yet if I specifically ask to build version.o immediately after running CMake for the first time, the header is not generated and the build errors out.

#7

I should add that I’m using CMake 3.19.3, so maybe you’re referring to changes on master instead.

#8

I was speaking about CMake targets dependencies.
For individual files, the explicit dependency is not specified, so it is normal that the generated header is not created if you ask directly to generate version.o.

If you want to be able to call generation of individual object files, you have to explicitly add the dependency over the generated header file:

set_property(SOURCE version.c PROPERTY OBJECT_DEPENDS "${CMAKE_BINARY_DIR}/${VER_FILE}")

But be aware that, because target CMakeFiles/bar.dir/depend is not called in this case, implicit dependencies for version.c can be missing or not up-to-date.