Add the capability of marking a target as NOTPARALLEL according to the documentation given by gnu-make. GNU make
The motivation of this feature request is having targets supporting several jobs and targets that should be serialized.
Interesting. For Ninja I’ve used JOB_POOLS to automatically constrain parallelism on targets known to use a lot of RAM when system RAM is below a threshold I experimentally determine in CMakeLists.txt.
I bring that up as maybe this potential target property could also be applied with Ninja generator and an internal job pool of size 1. Just a brainstorm, haven’t tried that.
Edit: as Brad notes, maybe add_dependencies could help?
What is the use case for that? If the targets have proper dependencies expressed, concurrent builds should not be a problem.
The use case for that is for instance the following:
Imagine we have a code Generator which does not support being called in a concurrent way.
TARGET_A1 (uses Generator)
TARGET_A2 (compiles code generated in TARGET_A1)
TARGET_A3 (is B executable)
TARGET_B1 (uses Generator)
TARGET_B2 (compiles code generated in TARGET_B1)
TARGET_B3 (is B executable)
If I run make all -j8 (Generator from TARGET_A1 and TARGET_B1) could be triggered in a concurrent way making the build fail.
If we could use the NOTPARALLEL feature from make for TARGET_A1 and TARGET_B1 we could prevent that scenario, right ?
The idea is to prevent the generator from called concurrently but allowing the compiler to compile the files using several jobs to build faster.
Are you invoking it via add_custom_command, or is it part of a toolchain backend?
We are invoking it via add_custom_command.
In our specific case we have solved the concurrency problem by calling the Generator using a script in between that checks if the Generator is running waits until it finished to proceed with the following Generator execution.
For reference, add_custom_command has a USES_TERMINAL option to give it exclusive use of the terminal when running with the Ninja generator. That is similar to this use case, but not the same.
For Makefile generators, there is no monolithic build graph. See here for a description of how multiple separate make tool invocations are used, and may be running concurrently. A .NOTPARALLEL mark in a single target’s build.make file will not protect against parallel execution of multiple make -f .../build.make processes.
calling the Generator using a script in between that checks if the Generator is running waits until it finished
That’s probably the only solution for your use case in general. It’s similar to how MSVC’s /FS flag synchronizes access to compiler-generated .pdb files through a mspdbsrv daemon during parallel compilation.
Sometimes especially template-heavy code can require a lot of memory, so that parallel builds fail due to memory issues. Restricting those targets to build not parallel can help.
Templates and CUDA code do this IME. However, JOB_POOLS is the better abstraction than “serialize this”. If/when the Makefiles generators do a monolithic graph, JOB_POOLS with size 1 could be supported.