Pull a custom target to earlier in the parallel build

I’ve a project with about 3500 targets, building with ninja in parallel on machines with between 4 and 8 cores.

One of these targets takes quite a long time, and that slows the entire build, since, for whatever reason, it usually ends up among the last of the targets to run.

It has the same list of dependencies as about 2500 other targets, so I would like to pull it forward, so that it will be among the first of those 2500 targets to run. It can consume a core for a long time while the other 2499 targets run on the remaining cores.

Note that naming it as a dependency to other targets so it is forced to run first is a BAD idea since it will block the other 2499 targets from running until it completes.

I’m not sure how to BEST do this…

Your help is appreciated!
Thanks!

May be you can try to define Job pool
https://cmake.org/cmake/help/v3.14/prop_tgt/JOB_POOL_COMPILE.html#prop_tgt:JOB_POOL_COMPILE

For this particular target so that it has its own pool while all other gets another pool. You put one core for this long target and number of core -1 for others.

Some of my targets require licenses, for which we only have a few. I do use JOB_POOL for those today to provide a LITTLE bit of intelligence so that one build won’t ask for more licenses that could be available. My point is that I know about pools and your idea is intriguing! Thanks!

If I define two pools, one for my long target, and one for everything else, would you expect cmake to schedule them independently?

What if I just defined a pool for the long target and left all the others in the “default pool”? Would it do the same thing?

Hmmm… I think I’ll just have to test it.

I think this needs ninja support. These issues seem related:

Once that is there, we can look at exposing knobs in CMake for them. Other generators will still be…less optimal still though.