Difficulty with add_executable() with generated files

Usage
1

The files does not get created in time to be included in add_executable()

What do I need to modify to get the files generation target run before add_executable so that ${generated_files} via file GLOB_RECURSE have files ?

include_directories(.)

set(TARGET_NAME vtkui)

file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/generatednodes)

add_custom_target(ALL # codegen
  COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/node_generation_main.py ${CMAKE_CURRENT_SOURCE_DIR}/nodes_definition.json ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/
  )

file(GLOB_RECURSE generated_files ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/*)

message("generated_files = ${generated_files}")

add_executable(${TARGET_NAME} WIN32
  VTKUIMainWindow.cpp
  abstractbase.cpp
  gcomponents.cpp
  #vtknodes/PNGReader.cpp
  #vtknodes/PNGWriter.cpp
  #vtknodes/GaussianSmooth.cpp
  main.cpp
  ${generated_files}
  # ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/generatednodes.h
  )

#add_custom_command(TARGET ${TARGET_NAME} PRE_BUILD
#  # OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/generatednodes.h
#  COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/node_generation_main.py ${CMAKE_CURRENT_SOURCE_DIR}/nodes_definition.json ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/
#  )

# add_dependencies(${TARGET_NAME} codegen)
2

The generator is called during build time, so the files do not even exist when CMake runs.
CMake needs to know which files it shall compile and link, so they need to explicitly listed. Otherwise it can’t generate the build rules for them.

3

May be you could have a look at this article:

which explains the issue of generated source files.

You’ll discover that using add_custom_command is better than add_custom_target provided that you can give the list of generated files ahead of time.

NB: if ${CMAKE_CURRENT_SOURCE_DIR}/node_generation_main.py is yours may be you can make it spit out the list of generated files in order to avoid the file(GLOB_RECURSE generated_files ... which is not executed soon enough (at configure time and not build time).

4

Yes, node_generation_main.py is my own code

Question
(1) Should I spit out the file names of the generated files as full path or relative path ? One line for each file or concatenated in a single line (as a CMake list string?) ?
(2) How should I then pass that information to add_executable ? Any pseudo code example ?

Cheers

5

I would generate a file that is of a known filename that #include’s the other files generated. This path can then be added to the source listing for the executable. If you do statically know the list of files that will be created, that would be better, but it seems that (based on the glob), the set of files depends on the content of that json file.

6

(1) since the generator is yours you can make it generate a list file that can be included using
https://cmake.org/cmake/help/latest/command/include.html, so the content of the generated file may defined a CMake variable, e.g. GENERATED_FILES which contains the list of generated file.

You can
include(generated_file.cmake OPTIONAL)
so that inclusion would not break if file is not there, but you’ll have to run cmake twice to get it right.

or you statically know the list of file (as @ben.boeckel said)

or you generate the list of file during configuration using https://cmake.org/cmake/help/latest/command/execute_process.html so that the to-be-include file is ensured to exists (provided execute_process happens before include).
In that last case you may have to rerun cmake manually when nodes_definition.json changes, unless
you add https://cmake.org/cmake/help/latest/prop_dir/CMAKE_CONFIGURE_DEPENDS.html property to nodes_definition.json file so that cmake will be rerun as soon as nodes_definition.json changes.

Now @ben.boeckel advised against this solution in the past: CMAKE_CONFIGURE_DEPENDS with newly created file so may be he can explain why.

7

Thank you @erk and @ben.boeckel

8

I would be interested in knowing your final solution @Nicholas_Yue

9

After reading Craig Scott’s post, I chose the “Generating Files At Configure Time” option.

As I am still in the prototyping/R&D phase of a project, I make the decision to priorities the time for something that is good enough for now.

file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/generatednodes)
include_directories(${CMAKE_CURRENT_BINARY_DIR}/generatednodes)

execute_process(COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/node_generation_main.py ${CMAKE_CURRENT_SOURCE_DIR}/nodes_definition.json ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/ )

file(GLOB_RECURSE generated_files ${CMAKE_CURRENT_BINARY_DIR}/generatednodes/*)

message("generated_files = ${generated_files}")

add_executable(${TARGET_NAME} WIN32
  VTKUIMainWindow.cpp
  abstractbase.cpp
  gcomponents.cpp
  main.cpp
  ${generated_files}
  )

When the prototype is approved with more resources, I will revisit this as I also need to look into packaging and installation via CMake.

Thank you for your support and interest.

Kind regards