Prevent rebuild every time for add_custom_target

Hello all,

When using add_custom_target with ALL option, I found that the target is built every time I run ninja.

Following the guidelines in https://discourse.cmake.org/t/add-custom-command-rebuilds-every-time/8213 (Unfortunately no solution is available), I figured out why this happens.

Below is the CMake command:

add_custom_target(${EXE_NAME} ALL
    COMMAND clang++-14 ${BC_FIN} -o ${EXE_FIN}
    DEPENDS ${BC_FIN}
)

And it generates the following ninja commands:

#############################################
# Custom command for CMakeFiles/SameDest

build CMakeFiles/SameDest | ${cmake_ninja_workdir}CMakeFiles/SameDest: CUSTOM_COMMAND SameDest-instrument.bc || librecord_branch.so
  COMMAND = cd <redacted>/build && clang++-14 <redacted>/build/SameDest-instrument.bc -o  <redacted>/build/SameDest

And ninja -d explain says:

ninja: Entering directory `build'
ninja explain: output CMakeFiles/SameDest doesn't exist
ninja explain: CMakeFiles/SameDest is dirty
ninja explain: SameDest is dirty

Obviously, the reason that causes rebuilding every time is CMakeFiles/SameDest does not exist, because my custom target does not output to that directory.

I manage to have a simple workaround by setting a link:

add_custom_target(${EXE_NAME} ALL
    COMMAND clang++-14 ${BC_FIN} -o ${EXE_FIN}
    COMMAND ln -s ${EXE_FIN} ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/${EXE_NAME}
    DEPENDS ${BC_FIN}
)  

Though everything works fine now, I’m curious about a clean way to build a custom target, i.e. without manually setting a soft link.


Thanks,
Fengkai

P.S. I use add_custom_target to invoke clang to create binary from bitcode.

Since bitcode is not recognized by cmake, add_executable will complain about no source file.

add_custom_command will not be executed if its OUTPUT is not a DEPEND of other targets.

I’ve made some further inspection, but that only makes me more confused:

When I use:

add_custom_command(OUTPUT ${EXE_FIN}
        COMMAND clang++-14 ${BC_FIN} -o ${EXE_FIN}
        DEPENDS ${BC_FIN}
)
add_custom_target(${EXE_NAME}-fin ALL
        DEPENDS ${EXE_FIN}
)

Then ninja.build would create a phony comand for this object:

#############################################
# Phony custom command for CMakeFiles/SameDest-fin

build CMakeFiles/SameDest-fin | ${cmake_ninja_workdir}CMakeFiles/SameDest-fin: phony SameDest || librecord_branch.so

I’m not sure what phony means, but I guess that is why in the case ninja does not try to inspect whether CMakeFiles/SameDest-fin is valid, and thus does not trigger a rebuild.

And that leads to the idiomatic way to do this in CMake: create a custom command to produce the output you want (let’s call it X), and then create a custom target that will DEPENDS on the custom command’s output (X). The target will always build, but since it won’t have a COMMAND, all it will do is bring its dependencies (i.e. X) up to date. If X doesn’t exist or is out of date, the custom command will build it. If X is up to date, nothing happens.

Hi Petr,

Thanks for your reply!

And that leads to the idiomatic way to do this in CMake

Did not know that. Thanks for letting me know!

It’s weird that the behavior of add_custom_target without specifying COMMAND is not documented, even so many people seem to use it in that way…

Can you please open an issue about the lacking documentation?

The linked discourse topic is just a question of some person, there’s no guideline in there.
That person already used a custom command, which you didn’t. And his cause of rebuilds is most likely the glob for source files, which seems to also include regularly changed .DS_Store files from MacOS.

Please rely on the CMake documentation:
The first paragraph of add_custom_target explains the behavior of always building and refers to add_custom_command as alternative.
The second paragraph of add_custom_command adds more details and even links an example where that combination is used.

Haven’t check the mailbox for days. I will do that after finalizing some details. Thanks for your advice!

The first paragraph of add_custom_target explains the behavior of always building and refers to add_custom_command as alternative.

I’ve already checked that. But I don’t think ‘always building’ equals to ‘force building’. The latter should be built whenever some changes have been made, while the latter suggests unconditional rebuild.

In my case, add_custom_target with specifying COMMAND will generate files under CMakeFiles/<target_name>, and if user command outputs elsewhere, that directory is never created and thus causing an unwanted force building even the source file is unchanged.

The second paragraph of add_custom_command adds more details and even links an example where that combination is used.

Thanks for the link, it does describe the joint use of add_custom_command and add_custom_target, though the example is not aimed for it. Honestly I only learned how to prevent race condition after reading that, but not creating a custom target without COMMAND.

I’m still quite new to CMake, so please correct me if I’m wrong. Thanks!

You have to think about, how all build tools work.
A build rule defines, how an output file is created, and which files are needed (so called dependencies).
The major task of build tools is to avoid executing all build rules, otherwise you could have written a shell script.
Makes usually do this by comparing the time stamp of the output file, with all dependencies. And only if a dependency is newer than the output, a rebuild of that rule is necessary. Ninja has some advanced techniques.

As add_custom_target explains, it has no output file by definition. In that case, a make cannot compare time stamps, it has to always execute the build rule.
So by definition it is always built, when the target is called.

For me, the word “force” implies, that you want to force the execution, even if it could be avoided. This is usually the case if you use e.g. make -B to force the re-creation of object files and executeables, even if they are up-to-date.

Edit: fixed the make -B instead of make -k

The -k flags just tell the tools to not stop scheduling new tasks when a command fails (those depending on the failed task are not scheduled however). Perhaps you mean the make -B flag (ninja has no equivalent)?

There’s no “forcing” done though. An add_custom_target(COMMAND) has no output and is therefore “never up-to-date” and needs to be run whenever the target is requested. Using the ALL flag makes this happen for “default” builds, but otherwise can be used to do things like a “deploy” command that always tries to deploy (whatever that means for the project at hand).

@ben.boeckel
It was too late at night. My mind mixed up the options. I usually use OpusMake where it’s -a and I knew it is a different letter on gnumake and picked the wrong one.
And that what I tried to explain, that only something like -B can be named as forced, but add_custom_target is simply built always by design.

Is there somewhere in CMake docs that say it is “forced”? Or is it just the comment above?

It’s not about the documentation, but this comment: