When running multiple cmake concurrently, "if (EXISTS xxx)" may be false while the file/folder exists.

My project have about 50 different targets with different configurations. I write a makefile to invoke multiple cmake to build all targets concurrently. With “make -j32 all”, I can make best use of my 16 core/32 threads CPU.

But I found it may fail on some targets. The failed targets are totally random. After some investigation, I found that the problem is caused by “if (EXISTS xxx)”. “if (EXISTS xxx)” may be false while the file/folder does exists.

This problem only happens on my Windows PC. The same commands is run on the gitlab CI runner, which is a Linux server, and it works fine.

When the number of jobs is reduced, the problem is less likely to happen, but still exists.

I wanted to replace “if (EXISTS xxx)” with other commands, but I found no suitable replacement.

My cmake version is 3.26.4 running on Windows 11 22H2. I tried cmake from both MSYS2/UCRT64 and https://cmake.org/download/, the problem is the same.

Have you tried adding a high level dependency with add_dependency for those targets to make sure that the get build in the correct order?

These different targets are independent. I use makefile to invoke multiple cmake instances, and these targets are built under different build directories.

To be more specific, the different targets are the firmware for different models of our products. I put the model-specific configurations in a xxxx.cmake file, and pass the model number to cmake as a CMake variable. The cmake script checks if xxxx.cmake exists, if true, the xxxx.cmake is included and if not, an error is reported.

The problem is the build may fail on some ramdon targets, but I’m sure all model-specific xxxx.cmake does exists.

It seems the problem is caused by relative path. The problem disappears if the checked path is absolute path.

Interesting sounds like the wd is in maybe in the wrong place? A bit hard too replicate ^^ but good to hear you have a workaround. For the absolute path are you using actual absolute paths or CMAKE_CURRENT_LIST_DIR or something?

The documentation for if(EXISTS) explicitly says the behavior is not well-defined if a relative path is given. You must use absolute paths with if(EXISTS).

1 Like

Ah, thanks! Docs:

Behavior is well-defined only for explicit full paths (a leading ~/ is not expanded as a home directory and is considered a relative path).