How can I be sure that a C++20 module is not re-compiled after a non-module code change?

Code
1

Hi Cmakers,

This is a question about compiling C++ code with modules.

I’ve built an example that is based on some existing C++ code I have. The class is called sm::range . I have made a CMakeLists to compile an executable target, use_range. The executable uses sm::range compiled as a module.

Here’s the CMakeLists.txt:

cmake_minimum_required(VERSION 3.28)

project(modules_test1 LANGUAGES CXX)

set(CMAKE_CXX_FLAGS “-g -Wfatal-errors”)

include_directories(BEFORE ${PROJECT_SOURCE_DIR})

set_source_files_properties (sm/range PROPERTIES LANGUAGE CXX)

add_executable(use_range use_range.cpp)
target_sources(use_range PUBLIC FILE_SET CXX_MODULES FILES sm/range)
target_compile_features(use_range PUBLIC cxx_std_20)

Find this at GitHub - sebsjames/modules_test1: Testing C++20 standard modules

I have adapted my C++ header file sm/range (original non-module code) with the necessary module; and export keywords.

The project compiles (after not a little work)!

However, I am not sure whether or not I am seeing the promised benefit of C++ modules in terms of build times, because if I trivially change the use_range.cpp code, and recompile, the recompilation seems to take as long as the initial compilation.

Here’s the process. Clean:

[12:40:13 build] cmake --build . --target clean [1/1] Cleaning all built files... Cleaning... 12 files.

Build:

[12:41:14 build] time cmake --build . --target all --verbose
Change Dir: '/home/seb/src/modules_test1/build'

Run Build Command(s): /usr/bin/ninja -v all
[1/6] /usr/bin/c++  -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/use_range.cpp -MT CMakeFiles/use_range.dir/use_range.cpp.o.ddi -MD -MF CMakeFiles/use_range.dir/use_range.cpp.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/use_range.cpp.o.ddi -fdeps-target=CMakeFiles/use_range.dir/use_range.cpp.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/use_range.cpp.o.ddi.i
[2/6] /usr/bin/c++  -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/sm/range -MT CMakeFiles/use_range.dir/sm/range.o.ddi -MD -MF CMakeFiles/use_range.dir/sm/range.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/sm/range.o.ddi -fdeps-target=CMakeFiles/use_range.dir/sm/range.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/sm/range.o.ddi.i
[3/6] /usr/bin/cmake -E cmake_ninja_dyndep --tdi=CMakeFiles/use_range.dir/CXXDependInfo.json --lang=CXX --modmapfmt=gcc --dd=CMakeFiles/use_range.dir/CXX.dd @CMakeFiles/use_range.dir/CXX.dd.rsp
[4/6] /usr/bin/c++  -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/sm/range.o -MF CMakeFiles/use_range.dir/sm/range.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/sm/range.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/sm/range.o -c /home/seb/src/modules_test1/sm/range
[5/6] /usr/bin/c++  -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/use_range.cpp.o -MF CMakeFiles/use_range.dir/use_range.cpp.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/use_range.cpp.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/use_range.cpp.o -c /home/seb/src/modules_test1/use_range.cpp
[6/6] : && /usr/bin/c++ -g -Wfatal-errors  CMakeFiles/use_range.dir/use_range.cpp.o CMakeFiles/use_range.dir/sm/range.o -o use_range   && :


real    0m0.641s
user    0m0.567s
sys     0m0.110s

Run the program:
[12:41:18 build] ./use_range
Range is [-1, 50]
[-1, 50]

Edit the program (changing a constant value 50.0f to 100.0f):

[12:41:21 build] emacs ../use_range.cpp

Re-build:

[12:41:35 build] time cmake --build . --target all --verbose
Change Dir: '/home/seb/src/modules_test1/build'

Run Build Command(s): /usr/bin/ninja -v all
[1/5] /usr/bin/c++  -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/use_range.cpp -MT CMakeFiles/use_range.dir/use_range.cpp.o.ddi -MD -MF CMakeFiles/use_range.dir/use_range.cpp.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/use_range.cpp.o.ddi -fdeps-target=CMakeFiles/use_range.dir/use_range.cpp.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/use_range.cpp.o.ddi.i
[2/5] /usr/bin/cmake -E cmake_ninja_dyndep --tdi=CMakeFiles/use_range.dir/CXXDependInfo.json --lang=CXX --modmapfmt=gcc --dd=CMakeFiles/use_range.dir/CXX.dd @CMakeFiles/use_range.dir/CXX.dd.rsp
[3/5] /usr/bin/c++  -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/sm/range.o -MF CMakeFiles/use_range.dir/sm/range.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/sm/range.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/sm/range.o -c /home/seb/src/modules_test1/sm/range
[4/5] /usr/bin/c++  -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/use_range.cpp.o -MF CMakeFiles/use_range.dir/use_range.cpp.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/use_range.cpp.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/use_range.cpp.o -c /home/seb/src/modules_test1/use_range.cpp
[5/5] : && /usr/bin/c++ -g -Wfatal-errors  CMakeFiles/use_range.dir/use_range.cpp.o CMakeFiles/use_range.dir/sm/range.o -o use_range   && :


real    0m0.617s
user    0m0.532s
sys     0m0.083s

Re-run to verify the program changed

[12:41:41 build] ./use_range Range is [-1, 100] [-1, 100]

Note 1: I see that there are 6 processes in the first build output and only 5 in the second. The process that does not appear in the second build is this one:

/usr/bin/c++ -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/sm/range -MT CMakeFiles/use_range.dir/sm/range.o.ddi -MD -MF CMakeFiles/use_range.dir/sm/range.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/sm/range.o.ddi -fdeps-target=CMakeFiles/use_range.dir/sm/range.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/sm/range.o.ddi.i

Which as far as I can tell is a preprocessing step that creates a preprocessed version of the code for the module.

I timed each of the 6 build steps and this is not one that takes a long time; it’s not an actual compilation.

Note 2: I wonder if I need to separate my sm/range into two files, say sm/range with the original C++ code implementation and sm/range.ixx (or .cppm) with the module interface?

So: Is there a reason that a recompile after editing use_range.cpp is not significantly faster than the clean-build?

Thanks for reading such a long post and best wishes,

Seb James

2

The command that isn’t run is the scan of the cm/range file (to get its module dependencies). Can you run ninja -d explain to see why it thinks it needs to recompile the sm/range file?

3

Thanks for the reply!

Here’s the same process using ninja -d explain --verbose:

[14:22:11 build] cmake --build . --target clean
[1/1] Cleaning all built files…
Cleaning… 12 files.

[14:23:20 build] ninja -d explain --verbose
ninja explain: depfile ‘CMakeFiles/use_range.dir/use_range.cpp.o.ddi.d’ is missing
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.ddi is dirty
ninja explain: depfile ‘CMakeFiles/use_range.dir/sm/range.o.ddi.d’ is missing
ninja explain: CMakeFiles/use_range.dir/sm/range.o.ddi is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.modmap is dirty
ninja explain: output CMakeFiles/use_range.dir/sm.range.gcm of phony edge with no inputs doesn’t exist
ninja explain: CMakeFiles/use_range.dir/sm.range.gcm is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o is dirty
ninja explain: use_range is dirty
[1/6] /usr/bin/c++ -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/use_range.cpp -MT CMakeFiles/use_range.dir/use_range.cpp.o.ddi -MD -MF CMakeFiles/use_range.dir/use_range.cpp.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/use_range.cpp.o.ddi -fdeps-target=CMakeFiles/use_range.dir/use_range.cpp.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/use_range.cpp.o.ddi.i
[2/6] /usr/bin/c++ -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/sm/range -MT CMakeFiles/use_range.dir/sm/range.o.ddi -MD -MF CMakeFiles/use_range.dir/sm/range.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/sm/range.o.ddi -fdeps-target=CMakeFiles/use_range.dir/sm/range.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/sm/range.o.ddi.i
[3/6] /usr/bin/cmake -E cmake_ninja_dyndep --tdi=CMakeFiles/use_range.dir/CXXDependInfo.json --lang=CXX --modmapfmt=gcc --dd=CMakeFiles/use_range.dir/CXX.dd @CMakeFiles/use_range.dir/CXX.dd.rsp
ninja explain: loading dyndep file ‘CMakeFiles/use_range.dir/CXX.dd’
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm.range.gcm is dirty
ninja explain: CMakeFiles/use_range.dir/sm.range.gcm is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o is dirty
ninja explain: use_range is dirty
[4/6] /usr/bin/c++ -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/sm/range.o -MF CMakeFiles/use_range.dir/sm/range.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/sm/range.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/sm/range.o -c /home/seb/src/modules_test1/sm/range
[5/6] /usr/bin/c++ -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/use_range.cpp.o -MF CMakeFiles/use_range.dir/use_range.cpp.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/use_range.cpp.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/use_range.cpp.o -c /home/seb/src/modules_test1/use_range.cpp
[6/6] : && /usr/bin/c++ -g -Wfatal-errors CMakeFiles/use_range.dir/use_range.cpp.o CMakeFiles/use_range.dir/sm/range.o -o use_range && :

[14:23:39 build] ./use_range
Range is [-1, 100]
[-1, 100]

[14:23:26 build] emacs ../use_range.cpp

[14:23:38 build] ninja -d explain --verbose
ninja explain: output CMakeFiles/use_range.dir/use_range.cpp.o.ddi older than most recent input /home/seb/src/modules_test1/use_range.cpp (1772461406087494134 vs 1772461417336117667)
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.ddi is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o is dirty
ninja explain: use_range is dirty
[1/5] /usr/bin/c++ -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -E -x c++ /home/seb/src/modules_test1/use_range.cpp -MT CMakeFiles/use_range.dir/use_range.cpp.o.ddi -MD -MF CMakeFiles/use_range.dir/use_range.cpp.o.ddi.d -fmodules-ts -fdeps-file=CMakeFiles/use_range.dir/use_range.cpp.o.ddi -fdeps-target=CMakeFiles/use_range.dir/use_range.cpp.o -fdeps-format=p1689r5 -o CMakeFiles/use_range.dir/use_range.cpp.o.ddi.i
[2/5] /usr/bin/cmake -E cmake_ninja_dyndep --tdi=CMakeFiles/use_range.dir/CXXDependInfo.json --lang=CXX --modmapfmt=gcc --dd=CMakeFiles/use_range.dir/CXX.dd @CMakeFiles/use_range.dir/CXX.dd.rsp
ninja explain: loading dyndep file ‘CMakeFiles/use_range.dir/CXX.dd’
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o.modmap is dirty
ninja explain: CMakeFiles/use_range.dir/sm.range.gcm is dirty
ninja explain: CMakeFiles/use_range.dir/sm.range.gcm is dirty
ninja explain: CMakeFiles/use_range.dir/use_range.cpp.o is dirty
ninja explain: CMakeFiles/use_range.dir/sm/range.o is dirty
ninja explain: use_range is dirty
[3/5] /usr/bin/c++ -I/home/seb/src/modules_test1 -x c++ -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/sm/range.o -MF CMakeFiles/use_range.dir/sm/range.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/sm/range.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/sm/range.o -c /home/seb/src/modules_test1/sm/range
[4/5] /usr/bin/c++ -I/home/seb/src/modules_test1 -g -Wfatal-errors -std=gnu++20 -MD -MT CMakeFiles/use_range.dir/use_range.cpp.o -MF CMakeFiles/use_range.dir/use_range.cpp.o.d -fmodules-ts -fmodule-mapper=CMakeFiles/use_range.dir/use_range.cpp.o.modmap -MD -fdeps-format=p1689r5 -x c++ -o CMakeFiles/use_range.dir/use_range.cpp.o -c /home/seb/src/modules_test1/use_range.cpp
[5/5] : && /usr/bin/c++ -g -Wfatal-errors CMakeFiles/use_range.dir/use_range.cpp.o CMakeFiles/use_range.dir/sm/range.o -o use_range && :

[14:23:41 build] ./use_range
Range is [-1, 50]
[-1, 50]

(edited to replace with ninja -d explain --verbose)

4

Strange. So yes, the scanning bits make sense. What doesn’t make sense is that sm/range.o.modmap is dirty after scanning to make use_range.cpp.o.ddi. The .modmap files are outputs of the cmake -E cmake_ninja_dyndep command which uses restat = 1 and should only update the outputs if they have different contents. Can you confirm that the _DYNDEP_ rule for use_range has restat = 1 on it (see CMakeFiles/rules.ninja). If so, can you compare the .modmap contents before and after the rebuild? Also its mtime if it is unchanged.

5

Thanks - I’ll do what you’ve asked and come back with results.

Independently, I was experimenting with an alternative compiler. I used gcc-14 in all the output that I have previously posted. I just tried clang-18 and this appears to compile in half the time after the ‘trivial edit’ and only carries out 4 steps. Could this be a compiler issue?

6

Here’s the CMakeFiles/rules.ninja:

# CMAKE generated file: DO NOT EDIT!
# Generated by "Ninja" Generator, CMake Version 3.28

# This file contains all the rules used to get the outputs files
# built from the input files.
# It is included in the main 'build.ninja'.

# =============================================================================
# Project: modules_test1
# Configurations: 
# =============================================================================
# =============================================================================

#############################################
# Rule for generating CXX dependencies.

rule CXX_SCAN__use_range_
  depfile = $DEP_FILE
  command = /usr/bin/c++ $DEFINES $INCLUDES $FLAGS -E -x c++ $in -MT $DYNDEP_INTERMEDIATE_FILE -MD -MF $DEP_FILE -fmodules-ts -fdeps-file=$DYNDEP_INTERMEDIATE_FILE -fdeps-target=$OBJ_FILE -fdeps-format=p1689r5 -o $PREPROCESSED_OUTPUT_FILE
  description = Scanning $in for CXX dependencies


#############################################
# Rule to generate ninja dyndep files for CXX.

rule CXX_DYNDEP__use_range_
  command = /usr/bin/cmake -E cmake_ninja_dyndep --tdi=CMakeFiles/use_range.dir/CXXDependInfo.json --lang=CXX --modmapfmt=gcc --dd=$out @$out.rsp
  description = Generating CXX dyndep file $out
  rspfile = $out.rsp
  rspfile_content = $in


#############################################
# Rule for compiling CXX files.

rule CXX_COMPILER__use_range_scanned_
  depfile = $DEP_FILE
  deps = gcc
  command = ${LAUNCHER}${CODE_CHECK}/usr/bin/c++ $DEFINES $INCLUDES $FLAGS -MD -MT $out -MF $DEP_FILE -fmodules-ts -fmodule-mapper=$DYNDEP_MODULE_MAP_FILE -MD -fdeps-format=p1689r5 -x c++ -o $out -c $in
  description = Building CXX object $out


#############################################
# Rule for compiling CXX files.

rule CXX_COMPILER__use_range_unscanned_
  depfile = $DEP_FILE
  deps = gcc
  command = ${LAUNCHER}${CODE_CHECK}/usr/bin/c++ $DEFINES $INCLUDES $FLAGS -MD -MT $out -MF $DEP_FILE -o $out -c $in
  description = Building CXX object $out


#############################################
# Rule for linking CXX executable.

rule CXX_EXECUTABLE_LINKER__use_range_
  command = $PRE_LINK && /usr/bin/c++ $FLAGS $LINK_FLAGS $in -o $TARGET_FILE $LINK_PATH $LINK_LIBRARIES && $POST_BUILD
  description = Linking CXX executable $TARGET_FILE
  restat = $RESTAT


#############################################
# Rule for running custom commands.

rule CUSTOM_COMMAND
  command = $COMMAND
  description = $DESC


#############################################
# Rule for re-running cmake.

rule RERUN_CMAKE
  command = /usr/bin/cmake --regenerate-during-build -S/home/seb/src/modules_test1 -B/home/seb/src/modules_test1/bgcc14
  description = Re-running CMake...
  generator = 1


#############################################
# Rule for cleaning all built files.

rule CLEAN
  command = /usr/bin/ninja $FILE_ARG -t clean $TARGETS
  description = Cleaning all built files...


#############################################
# Rule for printing all primary targets available.

rule HELP
  command = /usr/bin/ninja -t targets
  description = All primary targets available:

The only reference to restat appears to be to place the value of the variable RESTAT into restat, but there’s no other reference to RESTAT. But then there’s a file CMakeFiles/use_range.dir/CXX.dd which contains:

ninja_dyndep_version = 1.0
build CMakeFiles/use_range.dir/use_range.cpp.o: dyndep | CMakeFiles/use_range.dir/sm.range.gcm

build CMakeFiles/use_range.dir/sm/range.o | CMakeFiles/use_range.dir/sm.range.gcm: dyndep
restat = 1

Here’s the modmap changes:

After build, but before rebuild:

[17:30:36 use_range.dir] cat use_range.cpp.o.modmap
$root .
sm.range CMakeFiles/use_range.dir/sm.range.gcm

[17:31:07 use_range.dir] ls -l use_range.cpp.o.modmap
-rw-rw-r-- 1 seb seb 55 Mar  2 17:25 use_range.cpp.o.modmap

After an edit to use_range.cpp and a rebuild:

[17:32:48 use_range.dir] ls -l use_range.cpp.o.modmap
-rw-rw-r-- 1 seb seb 55 Mar 2 17:32 use_range.cpp.o.modmap
[17:32:53 use_range.dir] cat use_range.cpp.o.modmap
$root .
sm.range CMakeFiles/use_range.dir/sm.range.gcm

7

Yes, the CXX_DYNDEP__use_range_ rule is missing restat = 1. What version of CMake is this?

8

I’m using the one packaged on Ubuntu 24.04

[17:40:03 use_range.dir] cmake --version
cmake version 3.28.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

[17:40:16 use_range.dir] apt show cmake                                                               Package: cmake                                                                                        Version: 3.28.3-1build7                                                                               Priority: optional                                                                                    Section: devel                                                                                        Origin: Ubuntu                                                                                        Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>                                 Original-Maintainer: Debian CMake Team <pkg-cmake-team@lists.alioth.debian.org>                       Bugs: https://bugs.launchpad.net/ubuntu/+filebug                                                      Installed-Size: 37.4 MB                                                                               Depends: libarchive13t64 (>= 3.3.3), libc6 (>= 2.38), libcurl4t64 (>= 7.16.2), libexpat1 (>= 2.0.1), libgcc-s1 (>= 3.0), libjsoncpp25 (>= 1.9.5), librhash0 (>= 1.2.6), libstdc++6 (>= 13.1), libuv1t64 (>= 1.38.0), zlib1g (>= 1:1.1.4), cmake-data (= 3.28.3-1build7), procps
9

Found it. I fixed this in this MR which landed in 3.29.

10

Great! I can update my minimum version for cmake. Thanks for the help today! I’ll verify 3.29 works and then mark that the solution.

11

It was also backported to 3.28.4, so…Canonical really should pick up patch releases on a better schedule.

12

I can confirm that the correct rebuild behaviour is found in cmake 3.29.0 and in 3.28.5 (3.28.4 had the same behaviour as 3.28.3). Thanks again for the help!