Skip dependency checks in CMake + Ninja builds

Hello,

We have a very large code that uses CMake + Ninja which can take quite some time to build. We sometimes find ourselves doing printf style debugging within the code which can trigger rebuilding of large portions of the code. Our legacy Perl+Makefile based build system has an option to skip all dependency checks for this use-cases, build the single file that changed and then do the link.

Is there a way of accomplishing this in CMake + Ninja?

Thanks,

Galen

I don’t think so. There was talk somewhere about porting the /fast targets that the Makefiles generators make (which basically skip all dependency checking but are in the “keep both pieces” bucket when the build becomes inconsistent because of it), but I can’t find it right now.

Are you just running ninja or are you telling ninja to build just the target you’re interested in? You want to do the latter to minimise the amount of rebuilds while doing your dev iterations.

If you have lots of libraries involved in the target you are looking at, using shared libraries may speed things up. You could consider setting the LINK_DEPENDS_NO_SHARED target property to true, or more likely set it globally with the CMAKE_LINK_DEPENDS_NO_SHARED variable. That may reduce the amount of things that get rebuilt when you make changes.

If using static or object libraries, the OPTIMIZE_DEPENDENCIES target property might or might not help as well. Again, more likely you’d use the CMAKE_OPTIMIZE_DEPENDENCIES variable to enable this globally. This requires CMake 3.19 or later.

To add a few more details to @gshipman’s original question:

We have a large code base that is mostly Fortran 90 (and later) source files, that makes heavy use of modules. If a change is made to a Fortran source file that defines a module, then technically, any other source depending on that module file (directly or indirectly) needs to recompile. This can lead to a huge cascade of recompiles, especially if the original change is at a low level in the dependency tree. What we’d like to do is avoid all of those recompiles in the case of a trivial change that is known to not affect the generated module file, such as adding write statements for debugging.

We would still need to relink all of the binaries that depend on that source file, but we’re okay with the relinking steps since there generally aren’t nearly as many of those as there are of the recompiles. So I don’t think the LINK_DEPENDS_NO_SHARED or the other suggestions above will help with the problem we’re trying to solve. (If we were talking about C/C++ source file changes instead, those would be more relevant.)

One fix here is to get the compiler to either not rewrite a file with the same contents (in the vein of cmake -E copy_if_different). However, this requires restat = 1 and is generally not all that practical in the real world.

I think better is to instead enhance ninja here:

Getting CMake to help generate a build.ninja file that knows which edges need recompiled based on “trivial” changes is not trivial. We’re likely going to only be able to get “use the full dep tree” or “use no dep tree” (meaning manual relinking if needed) in practice.

@ben.boeckel Yes, this looks much more like what we need. I’ll go look more closely at the issues you linked.

And to clarify: we’re not looking for CMake to automatically decide whether a change is trivial, but to provide a setting that the user can set manually when (and only when) it’s appropriate.

@ben.boeckel as @cferenba indicates, we would be happy to have the ability to rebuild a single target and then force a relink of the application if that were an option (cmake --link)…

Ninja provides no build-time configuration or logic for such things. This would require a cmake -Dsome_flag . reconfigure to regenerate the build.ninja for any specific dependency weakening.

CMake does not, in general, know how to link a target. It knows how to generate the project files, but the actual link link is not known in Xcode and Visual Studio generators (the Makefiles and Ninja generators obviously do know the link line). I think the best CMake can do is provide /fast targets which ignore all non-object (or something; I forget the actual line that /fast cuts at) dependencies and then users can call ninja -j1 lib1/fast lib2/fast lib3/fast exe/fast (-j1 because otherwise those are going to run in arbitrary order and may not update properly).

1 Like