Cmake parallel build

Code
3

Thanks. Ive read it already. It says nothing of the way, I would like to trigger the parallel build.

I would like to force a parallel build for make (not ninja or sth else) over cmake

4

It may be that using cmake --build . -j is sufficient for the level of parallelism you are looking for.

5

It would have been, if the above custom_command is the way to achieve it ?

6

I just type cmake --build . -j at the command line when I want to build. You mentioned using make; you would just type make -j. There is no need for add -j via a custom command.

7

Right, I mentioned I do not want to type over the command line, but over a CMAKE_CUSTOM_COMMAND or whatever else possible. So, any suggestion of best practices of doing it, would be appreciated.

8

I don’t think you can do that but you can call make -j 5 or cmake --build -j5 the latter may use an environment var:
https://cmake.org/cmake/help/latest/envvar/CMAKE_BUILD_PARALLEL_LEVEL.html

9

Thanks, but as I mentioned already, Im looking for a solution, not using the command line.

10

How do you trigger your build?
you said:

so you use make but don’t call it?
I miss something.

11

I would like to set Generator Unix Makefiles, and than tell make to use the -j N parallel option.

1 Like
12

You do not really answer my question.

  • If you do invoke make then you may define make options in your environment MAKEFLAGS=-j5
  • if you do invoke cmake --build then you may CMAKE_BUILD_PARALLEL_LEVEL=5

In any case this is a user choice so such config ought to stay in user’s hand not in the CMakeLists.txt

You may invoke such parallel build using a custom_target if you prefer that:

add_custom_target(my_parallel_build 
                      COMMAND ${CMAKE_COMMAND} --build -j 5
                      WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
                      COMMENT "My parallel build with 5 cores")

and then you would invoke:
make my_parallel_build

13

I didnt know how to answer better your questions.

My goal is, that if a user sets to use a Unix Makefiles generator, and he if he forgets to set the parallel build flag, that than I add this flag over CMake.

The question is, if this is possible, and if yes, what is the way?

14

It is not currently possible to enforce parallelism of the build from within CMakeLists.txt.

1 Like
15

Except for setting the flag for Win VS. For Unix Makefile it is not supported.

16

If you are referring to /MP, that’s a flag of the compiler, not of the build tool.

GCC and clang simply don’t have such a flag. You could make a feature request on these projects.

1 Like
17

Wasnt aware of it. Part of the compiler? So the MS compiler is also a build tool and it parallelizes its compilation process? Interesting.

18

The build tool used by Visual Studio is MSBuild. But you can also use MSVC (the compiler) with Ninja, or NMake Makefiles.

When you pass /MP and several input files to MSVC, it “multiplies” itself to run in parallel. MSVC doesn’t track modification dates and dependencies, it is not a build tool.

1 Like
19

I have to ask again, why isnt it possible to set some BUILD_ARGS over the CMakeLists.txt?

It doesnt seem to me technically difficult, to set there a variable, which is expanded after cmake --build to .e.g. an additional --parallel N or --j N as I would like to in my case.

20

From the CMake documentation for --parallel [<jobs>], -j [<jobs>]:
The CMAKE_BUILD_PARALLEL_LEVEL environment variable, if set, specifies a default parallel level when this option is not given.

21

You are right as already given here: Cmake parallel build - #8 by erk
however @ProTur1 is seeking a CMake variable to be set in CMakeLists.txt not an environment variable.

I don’t personally know why this feature does not exist, may be you can try to implement it and see by yourself.
My very personal opinion is that I don’t want the provider of a project to enforce a level of parallelism on my build, if want to build with full parallelism I use ninja generator. If I want to chose a constrained parallelism I build with cmake --build --parallel <myChoice>.
The level of parallelism I want depends on various thing the developer of the project cannot know:

  • Am I building on my dev machine or in CI ?
  • Is my dev machine equipped with 4, 8 or 32 cores?
  • Is the build machine dedicated to that build or shall this build be a background activity which leaves enough computing power/RAM/disk activities etc… to other processes ?
    …

So I don’t know why this feature does not exist but my personal opinion is AFAIK I won’t need it anytime soon.

May be you could explain why, in your point of view this is a desirable/useful feature?

1 Like
22

From all the unknowns you mentioned, the cores can also be identified easily. From them, I would like to use only a part of it (although even if fully parallel, the OS scheduler should do its jobs well done as well).

Why it is useful?

  • for people who want to compile foreign projects, and don’t have time or don’t want to configure and type manually
  • for pushing to a build server, without command arguments for the build
  • for convenience
  • because it is easily possible to be realized (cmake offers anyhow a whole universe of functionality already)
1 Like