Simultaneous x86 and x86_64 build

Usage
1

Hi,

Thanks in advance for reply.

Let’s consider such situation of compiling very complex project.

cd [project]
mkdir x86
cd x86
cmake … -GNinja -A x86
ninja

cd [project]
mkdir x86_64
cd x86_64
cmake .. -GNinja -A x86_64
ninja

The compilation is happening in sequence - x86 → x86_64.

The issue with this approach revealed after we profiled CPU load time.
There is huge library linking in the middle of x86 compilation. It is single-threaded linking process.
All other cores of CPU are idle during this time.

Can we configure cmake to generate ninja build file, which includes both x86 and x86_64 definitions?

Something like:

cd [project]
mkdir build
cd build
cmake .. -GNinja -A [both_x86_AND x86_64]
ninja

It will allows us a better CPU utilization.

Is it possible?

2

No, CMake has just a single toolchain for any language. It’s a basic assumption that was baked in at the beginning and we’re stuck with it today. The closest you can get is universal binaries on macOS, but that requires compiler and toolchain support. Here, I would suggest using ExternalProject to run both of those in parallel at least.

3

Hy @ben.boeckel I realize this has been a fundamental cmake limitation for a while. But can’t gradual steps be taken to fix this problem eventually? Competing build systems are able to outperform cmake because of this limitation in some scenarios. Isn’t it worth the investment to start a roadmap to fixing this issue eventually?

Essentially something like this should be possible:
CMAKE_TOOLCHAIN_FILE=…/foobar.cmake
CMAKE_TOOLCHAIN_FILE_2=…/baz.cmake
CMAKE_TOOLCHAIN_FILE_3=…/coolcmake

I realize it’s a bit of work. But the fundamental concept/problem isn’t unfixable.

The bigger problem would perhaps be that older cmake would need to be updated to use generator expressions.

    if(CMAKE_SIZEOF_VOID_P EQUAL 4)
        ...
    elseif(CMAKE_SIZEOF_VOID_P EQUAL 8)
        ...
    endif()

But that is the same issue that occurs with users using CMAKE_BUILD_TYPE instead of using generator expressions.

4

No, it can’t be fixed in the existing CMake language. if (UNIX) and tons of other variables would need some context for which toolchain to query. Maybe it can be fixed with a declarative syntax on top of it, but that would still require tons of internal changes.

5

That makes sense. A multi-platform-build is an extremely out of scope feature.

However, a x86/x64 (ARM/ARM64) build with the same target platform is a much lighter request (I think).
A multi-architecture build feature (akin to multi-config).
The host is still the same.

6

Unless you run the configure stage several times. So just like an internal version of the currently manual superbuild solution…

7

Sure, but that’s essentially what you’re asking for at the core. Platform variables being different is no different a feature than CMAKE_SIZEOF_VOID_P being different for each toolchain… Any number of code logic variables could be different too (sizeof(time_t) for instance).

Then just do multiple configurations externally using ExternalProject or a script of some kind. The CMake command line would need to be much better (though the infrastructure has improved recently) to flow flags to different configurations, selecting build directories, etc. Even then, the builds will be disjoint anyways. How does one disambiguate between the various add_custom_target instances? How do you indicate that one build needs artifacts from another configuration (e.g., cross-compilation builds). Some of this might be answerable with strategies from Ninja Multi-Config, but other generators aren’t going to be happy with that kind of complexity.

Multiple toolchains for a single language in CMake is not doable today and has been rejected numerous times due to the amount of work it would take to upgrade CMake to rid itself of the assumptions laced throughout the codebase (not to mention all of the CMake code that will need fixed…FindMPI comes to mind shudder).

1 Like