# Race condition for -E create\_symlink duplicate calls

**URL:** <https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827>\
**Category:** Code\
**Created:** [April 4, 2023, 8:30pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827 "2023-04-04T20:30:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dwarwick](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dwarwick/32/2298_2.png) [@dwarwick](https://discourse.cmake.org/u/dwarwick)\
**Post date:** [April 4, 2023, 8:30pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/1 "2023-04-04T20:30:41Z")

</div>

CMake will happily write over the same symlink. If we have two targets, Target1 and Target2, which are in the same CMakeLists.txt (and therefore the same binary dir)

```auto
add_custom_command(
  TARGET Target1
  COMMAND ${CMAKE_COMMAND} -E create_symlink "${CMAKE_CURRENT_SOURCE_DIR}" "my_symlink2"
)

add_custom_command(
  TARGET Target2
  COMMAND ${CMAKE_COMMAND} -E create_symlink "${CMAKE_SOURCE_DIR}" "my_symlink2"
)

```

cmake will just override the existing one (whichever happens to run first) without a care in the world. No errors.

However, _very rarely_ (e.g. 1 in 100 builds on slow machines) we will see an error like this pop up:

`CMake Error: failed to create symbolic link 'C:/.../folder': Cannot create a file when that file already exists.`

which when running thousands of builds a day, leads to quite a few annoying spurious failures. What I believe is happening is that normally with all of the custom commands we have, the two separate create\_symlink calls aren’t executing at the same time when cmake runs them in parallel. However, in rare cases they will run at the same time and one will fail while another has a filesystem lock on the target symlink.

It’s possible to work around this in user code by either 1) running all create\_symlink calls serially by chucking them all into one add\_custom\_command (not great since it’s slower and forces you to pull a bunch of commands into one giant monolith), or 2) deduplicating the create\_symlink calls. That said, I think the more preferable solution would be if I could generate symlinks through cmake in a way that behaves like file(GENERATE where it automatically deduplicates redundant rules & throws an error if you have conflicting rules.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [April 4, 2023, 8:44pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/2 "2023-04-04T20:44:32Z")

</div>

(NOTE: Your `add_custom_command()` calls are missing a `POST_BUILD` keyword)

If you have both of those custom commands in a project, that project is malformed. You have defined two different targets that produce the same output. The only reason a build tool like Ninja doesn’t complain about it is because these commands also don’t specify what they create with a BYPRODUCTS keyword.

I’m wondering why this needs to be a build time rule at all though. Why can’t you create that symlink at configure time? It looks like there’s nothing about it that depends on something happening at build time.

---

<div class="post-metadata">

**Author:** ![dwarwick](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dwarwick/32/2298_2.png) [@dwarwick](https://discourse.cmake.org/u/dwarwick)\
**Post date:** [April 4, 2023, 9:17pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/3 "2023-04-04T21:17:19Z")

</div>

We run after configure since we need access to generator expressions like `TARGET_BUNDLE_CONTENT_DIR` which cmake does not seem to provide any configure-time equivalent to. If that weren’t an issue, then yes, I’d rather run this at configure time.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [April 4, 2023, 9:33pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/4 "2023-04-04T21:33:09Z")

</div>

Your earlier example where you created symlinks doesn’t use any generator expressions. It seems you may have over-simplified your real case when presenting a cut-down example here. Perhaps if you show a representative example without over-simplifying it, it may be possible to see what you’re really trying to do and whether there’s a cleaner, robust way to do it.

---

<div class="post-metadata">

**Author:** ![dwarwick](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dwarwick/32/2298_2.png) [@dwarwick](https://discourse.cmake.org/u/dwarwick)\
**Post date:** [April 4, 2023, 9:49pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/5 "2023-04-04T21:49:27Z")

</div>

```auto
get_target_property(is_bundle ${target} MACOSX_BUNDLE)
if(is_bundle)
    set(working_directory "$<TARGET_BUNDLE_CONTENT_DIR:${target}>/Resources")
else()
    set(working_directory "$<TARGET_FILE_DIR:${target}>")
endif()

add_custom_command(
    TARGET ${target}
    COMMAND ${CMAKE_COMMAND} -E make_directory ${working_directory}/Assets
    COMMAND ${CMAKE_COMMAND} -E create_symlink ${xmls_folder} "${working_directory}/Assets/Xmls"
)

```

We have a helper wrapper around add\_executable that this runs as part of since our executables require these resources in order to run. We have a few executables that share a directory with another though, so that results in this being executed against the same binary dir multiple times.

---

<div class="post-metadata">

**Author:** ![craig.scott](https://discourse.cmake.org/user_avatar/discourse.cmake.org/craig.scott/32/20_2.png) [@craig.scott](https://discourse.cmake.org/u/craig.scott)\
**Post date:** [April 5, 2023, 9:39pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/6 "2023-04-05T21:39:09Z")

</div>

Typically a bundle should be self-contained. It shouldn’t really contain symlinks to an area outside itself. For bundles, I’d recommend copying in the actual asset files into the bundle. You can do that using things like the `RESOURCE` target property and the `MACOSX_PACKAGE_LOCATION` source file property (use the latter if you need to preserve a path structure under the `Resources` directory). I think bundles are not your problem anyway, since each symlink will end up in a unique place (inside the bundle).

For non-bundle targets, you could use an `add_custom_command(OUTPUT...)` call to do the copying, then an `add_custom_target(...)` call that makes the custom target depend on the outputs from the `add_custom_command()` call. Then make all executable targets within your function depend on that custom target. You would have to come up with a way to generate unique names for the custom target, since you will do that in multiple directory scopes and target names must be unique. I’ll leave that as an exercise for you to work through.

Or, frankly, don’t try to automate this and let the project implement the copying where it needs it.

---

<div class="post-metadata">

**Author:** ![dwarwick](https://discourse.cmake.org/user_avatar/discourse.cmake.org/dwarwick/32/2298_2.png) [@dwarwick](https://discourse.cmake.org/u/dwarwick)\
**Post date:** [April 5, 2023, 9:53pm UTC](https://discourse.cmake.org/t/race-condition-for-e-create-symlink-duplicate-calls/7827/7 "2023-04-05T21:53:53Z")

</div>

This is for dev builds. Copying is too slow