# installing dependencies during build time with Ninja

**URL:** https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944
**Category:** Code
**Tags:** gen:ninja
**Created:** [March 12, 2021, 2:12am UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944 "2021-03-12T02:12:14Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Barry\_Revzin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/barry_revzin/32/1283_2.png) [@Barry\_Revzin](https://discourse.cmake.org/u/Barry_Revzin)
#### Post date: [March 12, 2021, 2:12am UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/1 "2021-03-12T02:12:14Z")

</div>

As part of our build process, we also install some dependencies that we have. Here is a short mostly-reproduction of what that looks like:

```cmake
cmake_minimum_required(VERSION 3.17)
project(wat)

# this is a path on my machine that the install
# command is allowed to write to, but I cannot
# I have sudo permission for that command only
set(FMT_MODULE /path/to/wherever/fmt-6.1.2)

# target to install fmt
add_custom_command(
  OUTPUT ${FMT_MODULE}/lib64/libfmt.a
  COMMAND flock /tmp -c 'sudo yum -y install fmt-6.1.2')
add_custom_target(install-fmt DEPENDS ${FMT_MODULE}/lib64/fmt.a)

# the imported fmt library
add_library(fmt-imported STATIC IMPORTED)
set_target_properties(fmt-imported PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES ${FMT_MODULE}/include
  IMPORTED_LOCATION ${FMT_MODULE}/lib64/libfmt.a
  )
  
# working around a cmake bug
add_library(fmt INTERFACE)
target_link_libraries(fmt INTERFACE fmt-imported)
add_dependencies(fmt install-fmt)

# just a dumb executable that needs fmt
add_executable(foo foo.cxx)
target_link_libraries(foo PUBLIC fmt)

```

In our setup, `yum install` will install that package in the path `${FMT_MODULE}` and while `yum` has permissions to do so (and I have permissions to run `sudo yum` in this context), I don’t, in general, have permissions to do anything in that path.

The above works totally fine if I use Makefile generators. If `fmt` is not installed, it will be installed (at build time), and everything just works exactly as I would like.

However, If I use Ninja, then this does not work. If `fmt` is not installed, then I get a permission denied failure because something is trying to run `mkdir` on that directory and I do not have permissions to do so. I’ve tried multiple permutations tried to indicate that the `add_custom_command` will end up creating that dependency but everything I’ve tried is a dead end, I just always get that “Permission denied” failure.

Is there a way to get this to work with Ninja? (@craig.scott)

---

<div class="post-metadata">

### Author: ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)
#### Post date: [March 12, 2021, 2:39am UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/2 "2021-03-12T02:39:46Z")

</div>

We need to narrow down where the “Permission denied” message is coming from. Are you able to narrow down the command in the `build.ninja` file that is being executed at the time that message is generated? You could try adding `--verbose` to your ninja command line.

---

<div class="post-metadata">

### Author: ![Barry\_Revzin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/barry_revzin/32/1283_2.png) [@Barry\_Revzin](https://discourse.cmake.org/u/Barry_Revzin)
#### Post date: [March 12, 2021, 2:44am UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/3 "2021-03-12T02:44:08Z")

</div>

Not exactly:

```auto
$ ninja --verbose
ninja: error: mkdir(/path/to/wherever/fmt-6.1.2): Permission denied
ninja: build stopped: .

```

I also don’t see `mkdir` anywhere in `build.ninja`, and am not familiar enough with Ninja do know which one of these rules tries to create that directory.

---

<div class="post-metadata">

### Author: ![Barry\_Revzin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/barry_revzin/32/1283_2.png) [@Barry\_Revzin](https://discourse.cmake.org/u/Barry_Revzin)
#### Post date: [March 12, 2021, 2:19pm UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/4 "2021-03-12T14:19:22Z")

</div>

I found `-d explain`, maybe this will help:

```auto
$ ninja -d explain -v
ninja explain: output /path/to/wherever/fmt-6.1.2/include/fmt/format.h of phony edge with no inputs doesn't exist
ninja explain: /path/to/wherever/fmt-6.1.2/include/fmt/format.h is dirty
ninja explain: output /path/to/wherever/fmt-6.1.2/include/fmt/core.h of phony edge with no inputs doesn't exist
ninja explain: /path/to/wherever/fmt-6.1.2/include/fmt/core.h is dirty
ninja explain: output /path/to/wherever/fmt-6.1.2 doesn't exist
ninja explain: /path/to/wherever/fmt-6.1.2/lib64/libfmt.a is dirty
ninja explain: CMakeFiles/install-fmt is dirty
ninja explain: /path/to/wherever/fmt-6.1.2 is dirty
ninja explain: /path/to/wherever/fmt-6.1.2/lib64/libfmt.a is dirty
ninja explain: CMakeFiles/foo.dir/foo.cxx.o is dirty
ninja explain: /path/to/wherever/fmt-6.1.2/lib64/libfmt.a is dirty
ninja explain: foo is dirty
ninja: error: mkdir(/path/to/wherever/fmt-6.1.2): Permission denied
ninja: build stopped: .

```

Also:

```auto
$ ninja -n
[0/3] Generating /path/to/wherever/fmt-6.1.2, /path/to/wherever/fmt-6.1.2/lib64/libfmt.aninja: error: mkdir(/path/to/wherever/fmt-6.1.2): Permission denied

ninja: build stopped: .

```

That description comes from this rule:

```auto
#############################################
# Custom command for /path/to/wherever/fmt-6.1.2

build /path/to/wherever/fmt-6.1.2 /path/to/wherever/fmt-6.1.2/lib64/libfmt.a: CUSTOM_COMMAND
  COMMAND = cd /home/brevzin/sandbox/wtf/build && flock /tmp -c 'sudo yum -y install fmt-6.1.2'
  DESC = Generating /path/to/wherever/fmt-6.1.2, /path/to/wherever/fmt-6.1.2/lib64/libfmt.a
  restat = 1

```

But the command there is… exactly what I asked for it to do - just install the thing, there’s no mkdir.

That last output is when I also added `${FMT_MODULE}/`, the directory, as an `OUTPUT`, which isn’t what I had in the OP. But the behavior is the same if I go back and remove it again.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [March 12, 2021, 4:35pm UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/5 "2021-03-12T16:35:28Z")

</div>

> [@Barry\_Revzin](#):
>
> But the command there is… exactly what I asked for it to do - just install the thing, there’s no mkdir.

Ninja will create directories for the output specified.

From [the manual](https://ninja-build.org/manual.html):

> [@Ninja manual](#):
>
> Output directories are always implicitly created before running the command that relies on them.

---

<div class="post-metadata">

### Author: ![Barry\_Revzin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/barry_revzin/32/1283_2.png) [@Barry\_Revzin](https://discourse.cmake.org/u/Barry_Revzin)
#### Post date: [March 12, 2021, 4:53pm UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/6 "2021-03-12T16:53:28Z")

</div>

I can see how that’s a convenient default, since that’s usually (nearly always?) what you would want to do anyway, but in this particular case it’s… extremely inconvenient.

Is there a workaround to tell Ninja to _not_ do this? Otherwise, we basically can’t use Ninja in this context. Like, I promise that the `COMMAND` here will actually create the directory and the file as it says it does.

---

<div class="post-metadata">

### Author: ![ben.boeckel](https://discourse.cmake.org/letter_avatar_proxy/v4/letter/b/ea5d25/32.png) [@ben.boeckel](https://discourse.cmake.org/u/ben.boeckel)
#### Post date: [March 12, 2021, 5:51pm UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/7 "2021-03-12T17:51:04Z")

</div>

No, there’s no workaround. Though I am wondering how you install a package to an arbitrary directory without a flag like `yum --root=` or something (I guess you could have a custom repo, but why not force it to be that one repo then?). Usually I would suggest to put such instructions into documentation rather than the build recipe and hard-fail if you can’t find `fmt` (e.g., this fails on non-Red Hat or newer CentOS/Fedora distros that lack `yum` for example). You also seem to be assuming x86\_64 and not supporting other arches which use `/usr/lib` instead. In addition, anyone without `sudo` rights is blocked and must modify the build system rather than being able to provide their own `fmt`. Maybe that’s fine in this instance, but it is weird to me.

---

<div class="post-metadata">

### Author: ![Barry\_Revzin](https://discourse.cmake.org/user_avatar/discourse.cmake.org/barry_revzin/32/1283_2.png) [@Barry\_Revzin](https://discourse.cmake.org/u/Barry_Revzin)
#### Post date: [March 12, 2021, 6:16pm UTC](https://discourse.cmake.org/t/installing-dependencies-during-build-time-with-ninja/2944/8 "2021-03-12T18:16:45Z")

</div>

> [@ben.boeckel](#):
>
> No, there’s no workaround.

I found a workaround. Instead of trying to use the dependency exactly where it will actually be installed, I’m adding a sym-link in my build directory and pointing the rest of my build through that sym-link. This way, when Ninja creates the directory, it’s inside of my build directory (which is fine). This works on both Ninja and Make.

```auto
set(SYM_LINKS ${CMAKE_BINARY_DIR}/sym_links)
set(FMT_LINK ${SYM_LINKS}/fmt-6.1.2)

add_custom_command(
    OUTPUT ${FMT_LINK}/lib64/libfmt.a
    COMMAND flock /tmp -c 'sudo yum -y install fmt-6.1.2'
    COMMAND mkdir -p ${SYM_LINKS}
    COMMAND rm -rf ${FMT_LINK}
    COMMAND ln -s ${FMT_MODULE} ${FMT_LINK}
    )
add_custom_target(install-fmt DEPENDS ${FMT_LINK}/lib64/libfmt.a)

```

> [@ben.boeckel](#):
>
> Though I am wondering how you install a package to an arbitrary directory without a flag like `yum --root=` or something

In reality the `yum` command is slightly different than what I showed, but not in a way that really affects the answer here - suffice it to say that I know where the package ends up being installed and what it looks like, and that the `yum` command has permissions to write in that location but I otherwise do not.
