add_custom_command behaviour different with make & ninja

Code
1

Consider this cmake file that uses a custom command to update the file /tmp/fred.out. The updating of this files has a DEPENDS on /tmp/fred.in

cmake_minimum_required(VERSION 3.17)
project(Hello_world VERSION 1.0)

set(INPUT_FILE  /tmp/fred.in)
set(OUTPUT_FILE /tmp/fred.out)


add_custom_target(fred ALL
    DEPENDS
        ${OUTPUT_FILE}
)

add_custom_command(
    OUTPUT
        ${OUTPUT_FILE}
    DEPENDS
        ${INPUT_FILE}
    COMMAND
        touch ${OUTPUT_FILE}
    COMMENT
        "Generating ${OUTPUT_FILE}"
)

Let’s create both the input and output files, and have the output file newer than the input file.

$ ll /tmp/fred*
-rw-r--r-- 1 xxx xxx 0 Jan 18 11:03 /tmp/fred.in
-rw-r--r-- 1 xxx xxx 0 Jan 18 13:11 /tmp/fred.out

Now build using make

$ make
[100%] Built target fred

$ ll /tmp/fred*
-rw-r--r-- 1 xxx xxx 0 Jan 18 11:03 /tmp/fred.in
-rw-r--r-- 1 xxx xxx 0 Jan 18 13:11 /tmp/fred.out

Note the output file has not been updated. This is what I’d expect given that we have a file-level dependency that isn’t stale.

Now do the same thing with ninja.

$ ninja 
[1/1] Generating /tmp/fred.out

$ ll /tmp/fred*
-rw-r--r-- 1 xxx xxx 0 Jan 18 11:03 /tmp/fred.in
-rw-r--r-- 1 xxx xxx 0 Jan 18 13:23 /tmp/fred.out

This time /tmp/fred.out has been updated. That was not what I’d expect to happen.

At the very least this is inconsistent behaviour, but which is deemed correct?

This what I’m running on a Linux setup

$ ninja --version
1.10.2

$ make --version
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010  Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ cmake --version
cmake version 3.22.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
2

Try using CMake for the touch command instead so the behavior works the same on all platforms.

add_custom_command(
    OUTPUT
        ${OUTPUT_FILE}
    DEPENDS
        ${INPUT_FILE}
    COMMAND
        COMMAND ${CMAKE_COMMAND} -E touch_nocreate ${OUTPUT_FILE}
    COMMENT
        "Generating ${OUTPUT_FILE}"
)
3

Thanks for the suggestion, but it didn’t make any difference – ninja still updated the file, while make did not.

Note that I’m running both tests on the same Linux host , so this isn’t a platform dependent issue.

1 Like
4

I guess the problem is that you assume that timestamp is the only condition for an up-to-date check, e.g.:
“Outputs implicitly depend on the command line that was used to generate them, which means that changing e.g. compilation flags will cause the outputs to rebuild.”
On first run, there is a file and unknown how it was created. Make just decides on timestamp but I guess that for ninja the command changed. That kind of makes sense to get to a known state from whatever was there before.

5

Thanks for that. I can understand both points of view, but the position at the moment is there is inconsistent behaviour. That is never good.

Is this behaviour down to how make & ninja work, or is it ultimately controlled by CMake ?

I did the equivalent experiment with add_executable to see if the dependency between the executable and its source file are handled in the same way. They are – ninja will rebuild the executable, while make will not.

This is causing us some pain because we need to create a file at configure time, but also have a custom target & command in place to spot any changes to the dependency to allow the output file to be regenerated.

Something along these lines

set(OUTPUT_FILE /tmp/fred.out)
set(INPUT_FILE  /tmp/fred.in)

execute_process(
    COMMAND
        touch ${OUTPUT_FILE}
)

add_custom_target(fred ALL
    DEPENDS
        ${OUTPUT_FILE}
)

add_custom_command(
    OUTPUT
        ${OUTPUT_FILE}
    DEPENDS
        ${INPUT_FILE}
    COMMAND
        touch ${OUTPUT_FILE}

    COMMENT
        "Generating ${OUTPUT_FILE}"
)

The problem with that is is the output file will get updated twice during a ninja build - first at configure time and then at some point during the build. The side effect of that means that if we immediately invoke ninja again it will find some targets out of date.

Using an add_dependency will probably make the problem go away, but the generation of the output file is buried in a cmake function that auto-generates the target name. Setting up the add_dependency will need to be abstracted into a wrapper function that knows how to map the output filename to its custom target. More complexity!

6

Ninja is no make. Please see the manual for some differences.
I would consider this command line check and the restat functionality superior over make.

Every build tool supported by CMake has its pros/cons. And not all features are supported in every build tool.

7

About your problem:
Having two different ways to create the same file is a bad design. It just worked somehow, because make does only a very simple time stamp check.

If you create that file at the configure/generate stage, you should add the dependent files to the directory property CMAKE_CONFIGURE_DEPENDS. So any build tool will call CMake configure automatically to re-generate that file first before building the rest. There’s no need for a custom command.

If the created file depends on some files modified by the build itself, then this disqualifies it to be created during configure/generate stage. Then it must always be created during build.