Ninja code generator does not build in correct order when using protobuf and custom_command

Code
,
1

I have a project which is using protobuf and a custom command.
However I get a compile error, because my custom_command is not executed in the correct order.
The problem occurs only with the ninja code generator. With make the code compiles fine.

Steps to reproduce:

cmake -S . -GNinja -B build
cmake --build build

../Main.cpp:2:10: fatal error: some_header.h: No such file or directory
    2 | #include "some_header.h"

For unknown reason I cannot upload my test case code in the forum…
So I uploaded it to a third party site. It is a minimal test case.

protobuf_test_case.tar.xz

Can anybody tell me how I can get my code working also with ninja?

2

can nobody help?

3

Hi,

It is easier if you show the code plain somewhere instead of an archive.
Your custom command looks like:

add_custom_command(
  TARGET MyProtoLib
  PRE_LINK
  COMMAND PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR} python3
          ${CMAKE_CURRENT_SOURCE_DIR}/test.py)

This cannot work when creating the header file like this. Ninja does not know what creates some_header.h (or if it exists somewhere in the include search path). And it is free to compile Main.cpp before create MyProtoLib static library.
You should prefer a custom command that is not bound to a target:

add_custom_command ( OUTPUT some_header.h
  COMMAND "${CMAKE_COMMAND}" -E env PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}
          python3 ${CMAKE_CURRENT_SOURCE_DIR}/test.py
  DEPENDS test.py
)

and then add some_header.h to the MyProtoLib or ConsumingService sources

target_sources ( MyProtoLib PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/some_header.h )
4

Hi Hendrik, thanks for your reply, this gives me a hint in the right direction.
Unfortunately I missed some important part of the test case. So the MyProtoLib creates some python files during compilation and this python file should be used when running test.py
so I added the missing import statement import foo_pb2.py
Thats why there is the dependency from the test.py to MyProtoLib

So the dependency tree is like that:

  1. Compile MyProtoLib
  2. Run test.py
  3. Compile ConsumingService

I adjusted the test case to reflect this, now the compilation fails again, also with make.

https://www.file-upload.net/download-14650441/protobuf_test_case2.tar.xz.html
PS: Why Can I still not upload file in the forum?

5

Hi @hsattler did you saw my answer?

6

some_header.h should be added as PRIVATE to MyProtoLib, not ConsumingService.
After that the dependency chain between Main.cpp.obj and some_header.h should be working

The add_custom_command for some_header.h needs an additions DEPENDS on ${CMAKE_CURRENT_BINARY_DIR}/foo_pb2.py to trigger it’s creating prior to running test.py.

7

@hsattler
I added:

target_sources ( MyProtoLib PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/some_header.h )

add_custom_command ( OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/some_header.h
  COMMAND "${CMAKE_COMMAND}" -E env PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}
          python3 ${CMAKE_CURRENT_SOURCE_DIR}/test.py
  DEPENDS test.py
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/foo_pb2.py
)

I get:

../src/Main.cpp:2:10: fatal error: some_header.h: No such file or directory
    2 | #include "some_header.h"

The python code itself will not be executed…so there is even no some_header.h created at all

8

I added
find_package(Python3 REQUIRED)
in the top-level CMakeLists.txt and used “${Python3_EXECUTABLE}” instead of plain python to make it work in Windows with python not in PATH.

Then I fixed the test.py to “import foo_pb2” (remove the .py suffix).
I then build with Ninja generator and it worked.

Since the test.py could not be run, there was not some_header.h. Always look at the first error message, not the last.

9 
  DEPENDS test.py
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/foo_pb2.py

should be

 DEPENDS test.py ${CMAKE_CURRENT_BINARY_DIR}/foo_pb2.py

see syntax of add_custom_command()