add_subdirectory with path outside project produces strange tmp directorys

Code
,
1

hello,
I have some project with Mbed that uses cmake now. There I have a project structure with shared interface libs that I want to have outside my main project.
That build and compiles when I use the add_subdirectory which needs the binary dir:

add_subdirectory("${MBED_PATH}" "${CMAKE_CURRENT_BINARY_DIR}/mbed-os")
add_subdirectory("../../custom_targets" "${CMAKE_CURRENT_BINARY_DIR}/custom_targets")
add_subdirectory("../../libs/lvglDriver" "${CMAKE_CURRENT_BINARY_DIR}/lvglDriver")
set(MICROPY_DIR true)           # force lvgl to create an interface lib
add_subdirectory("../../libs/lvgl" "${CMAKE_CURRENT_BINARY_DIR}/lvgl")

now I’m getting strange folder names for the object files in ./CMakeFiles/project.dir/
grafik

-- built: D:/Projects/Sn/LocalGit/Mbed6-Projects/apps/lvgl8-ST7735/cmake_build/STM32F407VE_BLACK/release/GCC_ARM/lvgl8-ST7735.hex
| Module                                          |           .text |       .data |          .bss |
|-------------------------------------------------|-----------------|-------------|---------------|
| 36e18ecf3f2b452c96fdc670bb9ca99b\cmsis          |       200(+200) |       0(+0) |         0(+0) |
| 72b785fc20e521e56bf35aca1777b714\TARGET_STM32F4 |           0(+0) |       0(+0) |         0(+0) |
| D_\Projects                                     | 124470(+124470) |   495(+495) | 42862(+42862) |
| [fill]                                          |       258(+258) |       5(+5) |       68(+68) |
| [lib]\c.a                                       |     8464(+8464) | 2108(+2108) |       58(+58) |
| [lib]\gcc.a                                     |       772(+772) |       0(+0) |         0(+0) |
| [lib]\misc                                      |       188(+188) |       4(+4) |       28(+28) |
| c0ecae1adab7d4c622418f1e546944e3\libs           |       800(+800) |       0(+0) |     476(+476) |
| c0ecae1adab7d4c622418f1e546944e3\mbed-os        |       326(+326) |       0(+0) |         0(+0) |
| de3935bbbceb118217cc343fec486e94\TARGET_STM32F4 |       638(+638) |       0(+0) |         0(+0) |
| e61ce3a1641df2c26379bb339841fd15\TARGET_STM     |     7584(+7584) |       4(+4) |         0(+0) |
| main.cpp.obj                                    |       988(+988) |       0(+0) |     180(+180) |
| Subtotals                                       | 144688(+144688) | 2616(+2616) | 43672(+43672) |

When I build the same project with Ubuntu, the foldernames are as expected. Is this a bug or is my config wrong?

2

It looks like something is doing md5 of some path to generate unique bits. But the behavior is new to me. @brad.king?

3

thanks for fast reply,
I’ve got one more clue. I have printed the binary dir with message("${CMAKE_CURRENT_BINARY_DIR}") and for windows I get:

D:/Projects/Sn/LocalGit/Mbed6-Projects/apps/test-new/cmake_build/STM32F407VE_BLACK/release/GCC_ARM

This is appended to the cmake_build dir which is generated by Mbed. And appending D:… to a path creates an invalid path. On the linux system, /home/me/… is appended, therefore it works.

so it looks my approach with
add_subdirectory("${MBED_PATH}" "${CMAKE_CURRENT_BINARY_DIR}/mbed-os")
is wrong.

Edit:
its still strange. I have changed the binary-path to xx-mbed-os, but the obj files are still placed into the numbered temp dirs. The xx- path is created, but there no object files.

grafik
grafik980Ă—760 70.4 KB

4

Indeed, this is quite strange. Does everything look OK if you do cmake --trace-expand? That is, there aren’t two absolute paths being put together in your source code. If that’s all fine, making a small reproducer test case (e.g., as a zip file) that demonstrates the problem would help us figure out the CMake side of things.

5

thanks, I will try to prepare. But this needs the mbed-tools installation.
I have seen Martin Kojital from ARM had recently also an issue, maybe you have already some mbed installation?

I tried to use some absolut path like c:/temp, but the same numbered dirs are created.

I have another screenshot from an Ubuntu system, there the output looks fine, but you see also that the path is concatenated. There the CMAKE_CURRENT_BINARY_DIR evaluates to:
/home/jojo/projects/Mbed6-Projects/apps/blinky/cmake_build/LPC1768/develop/GCC_ARM

grafik
grafik1850Ă—1175 283 KB

6

Well, I’d like to determine whether there’s a problem with CMake by showing that the bug is not in mbed-tools specifically. If there is evidence of mbed-tools putting paths together without checking the “right” side is absolute or not, that’s the bug. If it’s not doing something like that, then extracting whatever core it is doing into a small example would help track down what CMake is doing wrong here.

Basically, if this doesn’t happen with a dummy project that does add_subdirectory(../../elsewhere customdir), I’m hard pressed to be able to do anything about it since I don’t have any control over what mbed-tools is doing :slight_smile: .

7

this prints 10thousands of lines, I can’t redirect them into a file. How could the output be captured?

8

They’re printed to stderr, so 2>cmake-trace.log. I think that still works on Windows too.

1 Like
9

Processing: trace.zip…
I could not find the dir names in the output, also the binary dirs look fine.
Upload of tracefile is denied for me as new user.
ok, enough for today.
Thanks a lot for your help.

10

after lots of checking, I’ve done something simple: I have copied the project closer to the root to get shorter pathnames. And voila, it looks better:

grafik
grafik1161Ă—688 53.9 KB

but what I still not understand, when I provide a relative path to add_subdirectory for the binary path, it is still prepended by the full project path. This creates long pathnames that seem to be shortened, it happens in the cmake build configuration step.I couldn’t find some sort of concatenation in the Mbed cmake files.

edit:
there is a algorithm to shorten filenames: https://cmake.org/cmake/help/latest/variable/CMAKE_OBJECT_PATH_MAX.html
that explains the hash values in the filename

11

The logic in question is implemented in cmLocalGenerator::GetObjectFileNameWithoutTarget and cmLocalGenerator::CreateSafeUniqueObjectFileName. Everything you’ve observed is expected behavior.

12

yes, and it is even documented like this :slight_smile:
So thanks also for your confirmation. CMake is doing a great job, the object filename limit also prevents Ninja from failing. Ninja is still suffering from the older windows path limit, a PR to overcome this limit is stale for some time already.

Do you have also an explanation why the binary path in add_subdirectory contains a long path when only a short name is given? The temporary dir is created already in the build dir, so is this long appendix really neccessary?

13

CMake internally tracks all source/build directories using absolute paths. Providing a short name to add_subdirectory still references a directory that has an absolute path. The generators use that absolute path. Paths to object files are computed to prevent collisions among like-named sources while also not being dependent on the set of sources used. For source paths inside the top-level build or source directories, the object file paths are based on the relative locations. For source paths outside them, we have to use the absolute path to avoid ambiguity and collisions. That produces the long path.

1 Like