Race condition for -E create_symlink duplicate calls

CMake will happily write over the same symlink. If we have two targets, Target1 and Target2, which are in the same CMakeLists.txt (and therefore the same binary dir)

  TARGET Target1
  COMMAND ${CMAKE_COMMAND} -E create_symlink "${CMAKE_CURRENT_SOURCE_DIR}" "my_symlink2"

  TARGET Target2
  COMMAND ${CMAKE_COMMAND} -E create_symlink "${CMAKE_SOURCE_DIR}" "my_symlink2"

cmake will just override the existing one (whichever happens to run first) without a care in the world. No errors.

However, very rarely (e.g. 1 in 100 builds on slow machines) we will see an error like this pop up:

CMake Error: failed to create symbolic link 'C:/.../folder': Cannot create a file when that file already exists.

which when running thousands of builds a day, leads to quite a few annoying spurious failures. What I believe is happening is that normally with all of the custom commands we have, the two separate create_symlink calls aren’t executing at the same time when cmake runs them in parallel. However, in rare cases they will run at the same time and one will fail while another has a filesystem lock on the target symlink.

It’s possible to work around this in user code by either 1) running all create_symlink calls serially by chucking them all into one add_custom_command (not great since it’s slower and forces you to pull a bunch of commands into one giant monolith), or 2) deduplicating the create_symlink calls. That said, I think the more preferable solution would be if I could generate symlinks through cmake in a way that behaves like file(GENERATE where it automatically deduplicates redundant rules & throws an error if you have conflicting rules.

(NOTE: Your add_custom_command() calls are missing a POST_BUILD keyword)

If you have both of those custom commands in a project, that project is malformed. You have defined two different targets that produce the same output. The only reason a build tool like Ninja doesn’t complain about it is because these commands also don’t specify what they create with a BYPRODUCTS keyword.

I’m wondering why this needs to be a build time rule at all though. Why can’t you create that symlink at configure time? It looks like there’s nothing about it that depends on something happening at build time.

We run after configure since we need access to generator expressions like TARGET_BUNDLE_CONTENT_DIR which cmake does not seem to provide any configure-time equivalent to. If that weren’t an issue, then yes, I’d rather run this at configure time.

Your earlier example where you created symlinks doesn’t use any generator expressions. It seems you may have over-simplified your real case when presenting a cut-down example here. Perhaps if you show a representative example without over-simplifying it, it may be possible to see what you’re really trying to do and whether there’s a cleaner, robust way to do it.

get_target_property(is_bundle ${target} MACOSX_BUNDLE)
    set(working_directory "$<TARGET_BUNDLE_CONTENT_DIR:${target}>/Resources")
    set(working_directory "$<TARGET_FILE_DIR:${target}>")

    TARGET ${target}
    COMMAND ${CMAKE_COMMAND} -E make_directory ${working_directory}/Assets
    COMMAND ${CMAKE_COMMAND} -E create_symlink ${xmls_folder} "${working_directory}/Assets/Xmls"

We have a helper wrapper around add_executable that this runs as part of since our executables require these resources in order to run. We have a few executables that share a directory with another though, so that results in this being executed against the same binary dir multiple times.

Typically a bundle should be self-contained. It shouldn’t really contain symlinks to an area outside itself. For bundles, I’d recommend copying in the actual asset files into the bundle. You can do that using things like the RESOURCE target property and the MACOSX_PACKAGE_LOCATION source file property (use the latter if you need to preserve a path structure under the Resources directory). I think bundles are not your problem anyway, since each symlink will end up in a unique place (inside the bundle).

For non-bundle targets, you could use an add_custom_command(OUTPUT...) call to do the copying, then an add_custom_target(...) call that makes the custom target depend on the outputs from the add_custom_command() call. Then make all executable targets within your function depend on that custom target. You would have to come up with a way to generate unique names for the custom target, since you will do that in multiple directory scopes and target names must be unique. I’ll leave that as an exercise for you to work through.

Or, frankly, don’t try to automate this and let the project implement the copying where it needs it.

This is for dev builds. Copying is too slow