Some projects need to use ExternalProject because the sub-builds use a different architecture, different SDK, different build type or some other reason that means things can’t all be kept in a single CMake build. Ordinarily, Ninja is a good choice for using build resources efficiently, but it doesn’t handle this situation well. Ninja doesn’t support co-ordinated sharing of CPU resources across nested builds, unlike make which does. Currently, you end up having to make choices between potentially severe CPU over-commitment, or artificially limiting the parallelism to avoid that CPU over-commitment. The default behavior if you invoke Ninja on the top level build will be CPU over-commitment that gets worse as you add more external projects to the build because each external project thinks it has exclusive access to all the CPUs.
The problem is that a superbuild doesn’t let the external sub-build and the main parent build work together. I think Ninja doesn’t quite yet offer what is needed to do this well, but proposals like this one may offer a potential way forward. Should CMake people get involved in that discussion and see if we can find a way to allow a main build and an arbitrary number of sub-builds co-operate and share the CPU resources more efficiently? I don’t know what’s involved, but I wanted to open the discussion to see what people might already have explored and whether there’s an opportunity to improve this situation. @brad.king @ben.boeckel I think you have both been quite involved with Ninja, but I’m sure others are also interested and have experiences that are relevant here.
As extra motivation, this comes up in a few different scenarios, all of which are very real and I’m seeing more often:
- Projects that build for multiple embedded architectures and need to pull the results together into a single package or set of packages. Or perhaps be able to build, deploy and run a coherent set of multi-architecture artifacts.
- Android builds that want to produce binaries for different architectures and combine them in a single package.
- Apple platforms that target multiple architectures across multiple SDKs (macOS, iOS, each with their own multiple architectures). I suspect this will be very relevant for trying to create XCFrameworks, since we currently have no way of doing that given the combinations of SDK and architecture that may need to be included.
I’m sure there are more use cases, but I wanted to see if there’s anyone able and willing to investigate and maybe work on a solution to this.