# Incremental build with generated headers must run twice

**URL:** https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449
**Category:** Code
**Tags:** os:windows, comp:gcc, gen:ninja
**Created:** [February 12, 2023, 3:28pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449 "2023-02-12T15:28:42Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 12, 2023, 3:28pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/1 "2023-02-12T15:28:42Z")

</div>

I’m in the process of transitioning a legacy project into cmake, and I’ve hit a snag using a `custom_command` to generate header files.

On a clean build, everything works perfectly, the headers are generated and the code builds fine.

Incremental builds are a different story. If you modify an input file for the code generator, the next build will detect it and generate the headers just fine, but code that depends on the headers will _NOT_ be built. If you run the build again, this time it builds the code that depends on the generated headers. It feels like whatever checks for headerfile dependencies does it too early in the first run.

The relevant CMake code is below, any ideas what I’m doing wrong? I’ve been searching and trying things for a few days with no success.

```auto
################################################################################
## Code generation for autoset types.
################################################################################

#Location of the autoset header input files
cmake_path(SET autoset_input_dir "${PROJECT_SOURCE_DIR}/Project/Board-FunctionAgnostic")

# Location of XSL build script
cmake_path(SET autoset_generation_script "${PROJECT_SOURCE_DIR}/Project/Board-FunctionSpecific/xyzControl/GenerateAutosetHeaderFiles.ps1")
cmake_path(NATIVE_PATH autoset_generation_script native_autoset_generation_script)

add_custom_command(
    COMMAND pwsh -File "${native_autoset_generation_script}"
    DEPENDS
    "${autoset_generation_script}"
    "${autoset_input_dir}/generateTypesHeader.xsl"
    "${autoset_input_dir}/generateMachineTypesHeader.xsl"
    "${autoset_input_dir}/AutoSetTypes.xml"
    "${autoset_input_dir}/MachineTypes.xml"
    OUTPUT
    "${CMAKE_CURRENT_SOURCE_DIR}/W/AutoSetTypes.hpp"
    "${CMAKE_CURRENT_SOURCE_DIR}/MachineTypes.hpp"
    COMMENT "Generate Autoset header files"
)

################################################################################
## Board Function Agnostic Library
################################################################################

add_library(Board_FunctionAgnostic INTERFACE
    "${CMAKE_CURRENT_SOURCE_DIR}/W/AutoSetTypes.hpp"
    "${CMAKE_CURRENT_SOURCE_DIR}/MachineTypes.hpp"
)
# Sources are added here, but shouldn't be important
# Executables in this project use this library via "target_link_libraries" calls.

```

---

<div class="post-metadata">

### Author: ![leha-bot](https://discourse.cmake.org/user_avatar/discourse.cmake.org/leha-bot/32/921_2.png) [@leha-bot](https://discourse.cmake.org/u/leha-bot)
#### Post date: [February 12, 2023, 6:20pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/2 "2023-02-12T18:20:23Z")

</div>

Hello, Jeremy, may you try to use the  
[add\_custom\_target(Board\_GenerateAutosetFiles COMMAND …)](https://cmake.org/cmake/help/latest/command/add_custom_target.html) with the  
[add\_dependencies(Board\_FunctionAgnostic Board\_GenerateAutosetFiles)](https://cmake.org/cmake/help/latest/command/add_dependencies.html) ?

The second command adds the dependency from custom target to your library target.

Also you may add a custom target which would `DEPENDS` from your auto-generated files.

---

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 12, 2023, 7:22pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/3 "2023-02-12T19:22:30Z")

</div>

Trying that made things worse. Now the build always executes the generation script, but is only rebuilding the executable targets that use the library every other build.

```auto
################################################################################
## Code generation for autoset types.
################################################################################

#Location of the autoset header input files
cmake_path(SET autoset_input_dir "${PROJECT_SOURCE_DIR}/Project/Board-FunctionAgnostic")

# Location of XSL build script
cmake_path(SET autoset_generation_script "${PROJECT_SOURCE_DIR}/Project/Board-FunctionSpecific/xyzControl/GenerateAutosetHeaderFiles.ps1")
cmake_path(NATIVE_PATH autoset_generation_script native_autoset_generation_script)

add_custom_target(GenerateAutoset
    COMMAND pwsh -File "${native_autoset_generation_script}"
    DEPENDS
    "${autoset_generation_script}"
    "${autoset_input_dir}/generateTypesHeader.xsl"
    "${autoset_input_dir}/generateMachineTypesHeader.xsl"
    "${autoset_input_dir}/AutoSetTypes.xml"
    "${autoset_input_dir}/MachineTypes.xml"
# OUTPUT
# "${CMAKE_CURRENT_SOURCE_DIR}/W/AutoSetTypes.hpp"
# "${CMAKE_CURRENT_SOURCE_DIR}/MachineTypes.hpp"
    COMMENT "Generate Autoset header files"
)

################################################################################
## Board Function Agnostic Library
################################################################################

add_library(Board_FunctionAgnostic INTERFACE)
add_dependencies(Board_FunctionAgnostic GenerateAutoset)
# Sources are added here, but shouldn't be important
# Executables in this project use this library via "target_link_libraries" calls.

```

---

<div class="post-metadata">

### Author: ![leha-bot](https://discourse.cmake.org/user_avatar/discourse.cmake.org/leha-bot/32/921_2.png) [@leha-bot](https://discourse.cmake.org/u/leha-bot)
#### Post date: [February 12, 2023, 8:00pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/4 "2023-02-12T20:00:17Z")

</div>

oh, well, I don’t saw at first time that you generate the headers into the your SOURCE directory, but it’s recommended to use the BUILD directory for code generated files (e.g., `${CMAKE_CURRENT_BINARY_DIR}/W/AutoSetTypes.hpp` or simply `W/AutoSetTypes.hpp`) and add it into the `target_include_directories()`, so your source tree would be unaffected by changes from your custom command;

The CMake’s rule of thumb is to place any generated files into a build folder, e.g.:

1. [`file(GENERATE)`](https://cmake.org/cmake/help/latest/command/file.html?highlight=file#generate) manual states that “A relative path (after evaluating generator expressions) is treated with respect to the value of `CMAKE_CURRENT_BINARY_DIR`”;
2. The [`add_custom_command()`](https://cmake.org/cmake/help/latest/command/add_custom_command.html) manual states that “The path in the build directory is preferred unless the path in the source tree is mentioned as an absolute source file path elsewhere in the current directory.”

Also as of my anecdotal evidence, any changes in source directories causes some subtle bugs with build order. Hope this advice helps, and sorry for didn’t find the “high-level” manual/tutorial/best practice with this statement, may be @craig.scott or @brad.king knows better about this.

---

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 12, 2023, 8:23pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/5 "2023-02-12T20:23:10Z")

</div>

Unfortunately I’m stuck placing these files in the source directory. The project I’m working on now is a continuation of a much larger embedded software project that started in IAR. When any of the older IAR builds are run, they generate these same headers in the same location. I chose to place the code generated output at the same location in the source tree to avoid any ambiguity regarding what header files are being used.

The use of the `${CMAKE_CURRENT_SOURCE_DIRECTORY}` variable was to avoid relative paths, and get past the issues you mentioned above.

The strange thing is when I create small demo projects to model this (custom\_command to generate header files, interface library target that provides them, then another executable target that links to the library), everything works just fine, even when generating into the source tree.

---

<div class="post-metadata">

### Author: ![brad.king](https://discourse.cmake.org/user_avatar/discourse.cmake.org/brad.king/32/11_2.png) [@brad.king](https://discourse.cmake.org/u/brad.king)
#### Post date: [February 13, 2023, 3:30pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/6 "2023-02-13T15:30:42Z")

</div>

This case is expected to work in general and is covered in the test suite (other than using the source tree for the generated header). Try the following simple example:

```plaintext
$ cat ../CMakeLists.txt
cmake_minimum_required(VERSION 3.25)
project(Discourse7449 CXX)

add_custom_command(
  OUTPUT ${CMAKE_CURRENT_SOURCE_DIR}/header.hpp
  COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_SOURCE_DIR}/header.hpp
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/headerIn.txt
)

add_library(headers INTERFACE)
target_sources(headers PUBLIC FILE_SET HEADERS FILES header.hpp)

add_executable(main main.cpp)
target_link_libraries(main PRIVATE headers)

$ cat ../main.cpp
#include <header.hpp>
int main() { return 0; }

$ touch ../headerIn.txt

$ cmake --version
cmake version 3.25.2

$ cmake .. -GNinja

$ ninja
[3/3] Linking CXX executable main

$ ninja
ninja: no work to do.

$ touch ../headerIn.txt

$ ninja
[3/3] Linking CXX executable main

$ ninja
ninja: no work to do.

```

---

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 13, 2023, 11:41pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/7 "2023-02-13T23:41:49Z")

</div>

That demo works just fine. I’ve not attempted a toy demo like this yet with the cross compilation setup though. My only thought now is to try a skeletal embedded project so all the same tools are being used. Then I could at least share all the code, if that would help.

```auto
PS C:\Users\spams\Desktop\CMakeTest> cd build
PS C:\Users\spams\Desktop\CMakeTest\build> touch ../headerIn.txt
PS C:\Users\spams\Desktop\CMakeTest\build> cmake --version 
cmake version 3.25.0-rc2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
PS C:\Users\spams\Desktop\CMakeTest\build> cmake .. -GNinja   
-- The CXX compiler identification is GNU 9.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/cygwin64/bin/c++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/spams/Desktop/CMakeTest/build
PS C:\Users\spams\Desktop\CMakeTest\build> ninja
[3/3] Linking CXX executable main.exe
PS C:\Users\spams\Desktop\CMakeTest\build> ninja
ninja: no work to do.
PS C:\Users\spams\Desktop\CMakeTest\build> touch ../headerIn.txt
PS C:\Users\spams\Desktop\CMakeTest\build> ninja
[3/3] Linking CXX executable main.exe
PS C:\Users\spams\Desktop\CMakeTest\build> ninja
ninja: no work to do.
PS C:\Users\spams\Desktop\CMakeTest\build>

```

---

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 16, 2023, 1:27am UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/8 "2023-02-16T01:27:41Z")

</div>

Alright, I have a minimal demo project up on github:

> **[GitHub - JeremyRodencal/CMakeDoubleBuildDemo: Quick repo to show an odd code...](https://github.com/JeremyRodencal/CMakeDoubleBuildDemo)**
>
> Quick repo to show an odd code generation double build when cross compiling on windows. - GitHub - JeremyRodencal/CMakeDoubleBuildDemo: Quick repo to show an odd code generation double build when c...

This double build issue _only_ happens when I’m cross compiling from windows, but does _not_ occur if I’m compiling with mingw64 for the host machine. It does _not_ occur when compiling for the host or cross compiling on linux.

The bug doesn’t trigger with the simple example given by @brad.king at all. Once I stripped down to a rough model of what my project was doing the issue presented itself.

I tried a few versions of gcc arm cross compiler on windows, but all of them yielded the same results.  
The compilers were obtained from here: [Arm GNU Toolchain Downloads – Arm Developer](https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads).

---

<div class="post-metadata">

### Author: ![ConcernedPanda](https://discourse.cmake.org/user_avatar/discourse.cmake.org/concernedpanda/32/3160_2.png) [@ConcernedPanda](https://discourse.cmake.org/u/ConcernedPanda)
#### Post date: [February 26, 2023, 11:33pm UTC](https://discourse.cmake.org/t/incremental-build-with-generated-headers-must-run-twice/7449/9 "2023-02-26T23:33:09Z")

</div>

The windows build of arm GCC (12.2 rel1 was the newest version I tested) is the issue here. The dep files it generates have file paths converted to lower case. That throws off ninja and causes the need for a double build. Hacking up a build of ninja to restore the correct path casing made everything work correctly.

Mingw outputs correct paths in dep files, so that was why it worked fine when I wasn’t cross compiling.