# Control build parallelism within ExternalProject\_Add

**URL:** https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864
**Category:** Code
**Created:** [June 14, 2022, 9:59am UTC](https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864 "2022-06-14T09:59:02Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![bthomasz](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ecc23a/32.png) [@bthomasz](https://discourse.cmake.org/u/bthomasz)
#### Post date: [June 14, 2022, 9:59am UTC](https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864/1 "2022-06-14T09:59:02Z")

</div>

Hi,

I am working on a project to build&package 3rd party dependencies for an application, by calling ExternalProject\_Add exclusively. This runs in linux vm-s using both Ninja and Unix Makefile generators.

The problem:

- Ninja automagically detects parallelism. Well, in practice it detects the number of host CPUs instead of the ones allocated to the VM → overcommitting and competing for resources → very slow build
- Unix Makefile runs in serial by default

What I tried:

- added parallel level, no effect:

```auto
      ExternalProject_Add(
      CMAKE_ARGS -DCMAKE_BUILD_PARALLEL_LEVEL=${NCORES} -DENV{CMAKE_BUILD_PARALLEL_LEVEL}=${NCORES}
      ...
      )

```

- also tried to set the job pool for Ninja, by post-adjusting the target, also no effect:

```auto
      set_property(GLOBAL PROPERTY JOB_POOLS builder_pool=${NCORES})
      ExternalProject_Add(${target_name} ...)
      set_property(TARGET ${target_name} PROPERTY JOB_POOL_COMPILE builder_pool)
      set_property(TARGET ${target_name} PROPERTY JOB_POOL_LINK builder_pool)

```

So the question is, how can I override the number of cores for the individual builds? I only need to support Unix MakeFile and Ninja generators, so an if (CMAKE\_GENERATOR EQUALS “Ninja”) … else … kind of solution is still acceptable for me (but something generic would be OK too 😉 ).

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [June 14, 2022, 11:20am UTC](https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864/2 "2022-06-14T11:20:02Z")

</div>

It’s not really supported well unfortunately. I’ve done some work to make this happen in [this project](https://gitlab.kitware.com/paraview/common-superbuild), but it’s never been upstreamed.

See [this code[([https://gitlab.kitware.com/paraview/common-superbuild/-/blob/master/cmake/SuperbuildExternalProject.cmake#L16-L32](https://gitlab.kitware.com/paraview/common-superbuild/-/blob/master/cmake/SuperbuildExternalProject.cmake#L16-L32)) (and the variable uses later in the file) for how it’s done. You can probably get away by manually specifying `BUILD_COMMAND` or something.

---

<div class="post-metadata">

### Author: ![bthomasz](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ecc23a/32.png) [@bthomasz](https://discourse.cmake.org/u/bthomasz)
#### Post date: [June 14, 2022, 1:11pm UTC](https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864/3 "2022-06-14T13:11:15Z")

</div>

At the end figured out something simpler. One can chain the cmake commands to set CMAKE\_BUILD\_PARALLEL\_LEVEL and this works (maybe I am doing something wrong but passing --parallel 2 is failing with argument “2” not understood).

```auto
ExternalProject_Add(
...
BUILD_COMMAND ${CMAKE_COMMAND} -E env CMAKE_BUILD_PARALLEL_LEVEL=${NCORES} ${CMAKE_COMMAND} --build .
)

```

This is not a general solution but works for me because it’s always --build . for my project. TBH, things would be easier if the commands could be modified for example with ExternalProject\_Get\_Property/ExternalProject\_Set\_Property. Something like this:

```auto
# Do not specify BUILD_COMMAND, so the default will be generated
ExternalProject_Add(${target_name} ...)
ExternalProject_Get_Property(${target_name} BUILD_COMMAND)
ExternalProject_Set_Property(${target_name} BUILD_COMMAND ${CMAKE_COMMAND} -E env CMAKE_BUILD_PARALLEL_LEVEL=${NCORES} ${BUILD_COMMAND})

```

---

<div class="post-metadata">

### Author: ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)
#### Post date: [June 14, 2022, 11:07pm UTC](https://discourse.cmake.org/t/control-build-parallelism-within-externalproject-add/5864/4 "2022-06-14T23:07:40Z")

</div>

See also [Efficiency issues with ExternalProject sub-builds under Ninja](https://discourse.cmake.org/t/efficiency-issues-with-externalproject-sub-builds-under-ninja/3241)