TARGET_OBJECTS macro, missing character in one of the object files

Hello guys,

I am using cmake 3.20.5 and we have created a quite big build system for windows / Linux and Mac.
One of our library (OBJECT library) includes a big number of object files and when using TARGET_OBJECTS macro on it , from time to time - I can say even rarely, one of the object file in the list is missing a character, causing the build to fail.

This happens quite randomly and is also rare - so difficult up to impossible to supply a reproduction of the problem.

Does it sound familiar to someone, workaround? fix?

Thanks,
Serge

With macro I assume you mean the generator expression, right?

The content is always written to a file during the CMake call; during the build this information is only used.
So if the build fails, you can easily check if that character is really missing in the CMake output or not.

Does it happen on the same PC, then it may be simply RAM or disc problems. Other random problems mainly involve “outer” parallelization, like someone (or CI system) is calling the same build 2 times.

Hi Joseph,

Yes this is the generator expression: $<TARGET_OBJECTS:target>.
Following your recommendation I have been looking into the generated files and it looks weird. Let me elaborate and maybe we can drill down to the root cause.

We are using the generation expression to dump the object file list into a file in the following way:

add_command(

COMMENT “${CMAKE_COMMAND} -E echo $<target_objects:XXX> > some_file.txt”
COMMAND ${CMAKE_COMMAND} -E echo $<target_objects:XXX> > some_file.txt
… )

The resulting build log file shows the above command with the right file list. This is true for the .vcxproj generated. But some_file.txt includes the exact same list but one object file name is being missing a character.
It seems to happen on a specific machine and I am not sure it does not happen on others.

Would you say that “${CMAKE_COMMAND} -E echo” may have an issue with very long lists?
Is there an alternative way to dump this list to a file?

Thanks,
Serge

One correction:

not target_objects above but TARGET_OBJECTS

Out of curiosity, why not use file(GENERATE) instead? This doesn’t need to be done at build time.

file(GENERATE OUTPUT some_file.txt CONTENT "$<TARGET_OBJECTS:XXX>")

The most problems, like “command line too long” or “special unquoted character on command line” are fully reproducible.
And that’s why you should write the file as Scott suggested. This avoid any of such problem in general.

But it’s still not an explanation for missing a single character. If the problem disappears by simply starting the same build a second time (after a clean), then I’m quite sure that it’s a hardware problem (RAM, CPU or …). You can try to do a RAM and CPU Load test (e.g. MemTest86 and Prime95) on that PC.

First thanks a lot for your suggestion above which seems to work.
Somehow file(GENERATE …) does not have this issue.

Although file(GENERATE OUTPUT some_file.txt CONTENT …") is a bit tricky since I got some errors saying:
“Evaluation file to be written multiple times with different content”

I had to play a little with the file name and have $<CONFIG> in it otherwise CMake is not ready to accept. Finally it passed.

Answering to Joseph: the problem with the missing character is consistent but only with specific machines - it seems that “windows Server 2016” is having that issue but other machines do not - so I do not think this is a RAM or hardware issue since if it was, then the issue would not happen with that machine repetitively and at the same place: this problem appears randomly on environment / hardware but it occurs again and again once you have a machine with the issue and at the same place.

Really weird and difficult to imagine a bug with a behavior like this …

Thanks a lot for your help,
Serge

Is the missing character always the first one of a file name or directory name in a path? I’m thinking you might be seeing cases where backslashes are used in paths, but CMake is treating them as escapes. If those correspond to recognised escape sequences (\n, \t, \r and so on), that may explain why those particular characters would be missing in the output.

Note that this isn’t describing a bug in CMake, this would typically be a bug in the way you’ve defined the project or provided information to CMake.

@craig.scott But I guess this should be a reproducible problem.

@serge As you recognized now, that you’re writing different contents to the same file, we can come to the other root cause for randomness: parallelization.
Do you build the different configuration in parallel? If yes, the file might be overwritten by one config build while read by the other one.
If that’s really the case, the old way with echo should also be stable as soon as you use distinct file names.

@craig.scott - Sometimes we get …\xxx.ob instead of …\xxx.obj which hits the last character of a specific object file in a long list of object files. Some other times we got …\director_name… instead of …\directory_name… so I am not sure it is located at some special file path name and is related to escape characters.

@Josef - This is interesting and I am not sure we are doing the right thing. This is windows and we build a specific configuration, say Release. The generation phase is done with:
“cmake -H. -Bbuild -DCMAKE_BUILD_TYPE=Release -G “Visual Studio 16 2019” -A Win32”
while the build phase is done with:
“cmake --build build --config Release --target install”

I saw indeed that file(GENERATE OUTPUT “list.$.str”…) creates two files: list.Debug,str and list.Release.txt which surprised me since we have set release above … Could there be a miss in the way we invoke CMake?

Thanks.

Above, I meant file(GENERATE OUTPUT “list.$<CONFIG>.str”…)
Somehow the $ caused CONFIG to be hidden