Advice on how to compile same framework with n files f_i each generating m_i targets.

Hello,

Until now, I was using Makefiles, but I have decided to use CMake for the first time for a new project. The project is a framework for studying branch predictors. It should allow to generate different binaries using the source code of different branch predictors. For example, binaries that run a branch predictor over a trace and output the misprediction results, and others that compare more than one predictor. However, being a new user, I am not fully aware of what would be the best way to obtain different targets based on the same files from the framework and different source files for different predictors, while avoiding unnecessary recompilation.

The structure of the project is the following.

project
├ bp/
│ ├── predictor_a.hpp, predictor_a.cpp // sources for predictor a
│ └── predictor_b.hpp, predictor_b.cpp // sources for predictor b
├ include/
│ ├── branch_type.hpp
│ ├── trace_reader.hpp
│ ...
└ src/ -- folders with the sources of the framework.
  ├── simulator.cpp
  ├── comparator.cpp
  ├── trace_reader.cpp
  ...

The files for each predictor declare a class with a predict function. The source files simulator.cpp and comparator.cpp contain a main() function. They must instantiate some number of predictors and use them. Therefore, they must be compiled and linked with different predictors for creating different targets. Other source files, like trace_reader.cpp, need to be compiled just once because they do not depend on the predictor.

Currently, my CMakeList is something like the following.

cmake_minimum_required(VERSION 3.10)

project(SimpleSimulator
  VERSION 0.1
  LANGUAGES CXX
)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
include_directories(include bp)
set(CMAKE_CXX_FLAGS
  "${CMAKE_CXX_FLAGS} -Wall -O3 -march=native -mtune=native -flto"
  CACHE STRING "Set C++ Compiler Flags"
  FORCE
)
set(CMAKE_LINK_OPTIONS
  "${CMAKE_LINK_OPTIONS} -flto"
  CACHE STRING "Set Linking Options"
  FORCE
)

function(add_simulator name header class)
  add_executable(${name}
    src/simulator.cpp
    src/branch_type.cpp
    src/trace_reader.cpp
  )
  // by using some preprocessor definitions, we are able to compile the same file with different bps.
  target_compile_definitions(${name}
    PUBLIC
      BP_HEADER="${header}"
      BP_CLASS=${class}
  )
endfunction()

add_simulator(ashare ashare.hpp Ashare)
...

Basically, I wrote the simulator.cpp file using the macros BP_HEADER and BP_CLASS which are provided during compilation and created the CMake function add_executable which allows to declare a new target using a simulator.

The problem with this configuration is that the object files like trace_reader.cpp are compiled for each target (while they could be reused). Additionally, I am not sure that including the dependency in the simulator.cpp file through a macro is the best option.

How would you handle this two problems?

Thank you in advance,

Some notes:

  • your flag manipulation will add an instance of the flags you want to the command line on every reconfigure
    • for -flto, use INTERPROCEDURAL_OPTIMIZATION instead
    • -O3 and -mtune should just be appended to CMAKE_CXX_FLAGS locally; no need to put it into the cache (same with -Wall actually)

Factor the common files out into a static library that is reused by each executable.

Or, into an OBJECT target that’s linked into each executable.

I do something very much like that for our unit tests, which are based on Catch2. The unit tests for each class are built into their own executable, so there are a ton of them. Building the Catch2 testrunner is slow as hell, especially on Windows under MSYS2, so compiling the same source file 36 times was really dragging down build times.

Sp now, our unit tests CMakeLists.txt builds the Catch2 portion just once, and links the object into all of the test executables:

# Create object library for test executable main(),
# to avoid recompiling for every test
add_library(catch-main OBJECT catch_main.cpp)
target_link_libraries(catch-main PUBLIC Catch2::Catch2)

foreach(tname ${UNIT_TESTS})
  add_executable(${tname}-test
    ${tname}.cpp
    $<TARGET_OBJECTS:catch-main>
  )
endforeach()

To be honest, I would have to read the documentation a little bit more to understand the difference. Right now I am learning things the moment I need them, but I have to read a proper guide.

The problem with this is that at some point the code inside main needs to get an instance of the user class. I am not sure on how to do this. I think the way the Catch library works is by registering each test in some kind of list, is that right? What is the best way to have the different “mains” get the instances of branch predictors they need? Maybe the question should be asked in a C++ forum instead of a CMake forum.

Giving some more concrete example that can be played around with would also help.

Here’s a simple example of how you can do it, using polymorphism and a factory function to create the instances. main() never sees anything but a pointer to the base class, so it’s completely branch-agnostic.

// testers.h
//
#pragma once
#include <string>

class ThingTester {
	public:
		ThingTester() = default;
		virtual void do_test() = 0;
		virtual ~ThingTester() = default;
};
ThingTester* factory();
// testers.cpp
//
#include <algorithm>
#include <iostream>
#include <string>
#include "testers.h"

#if TEST_INTS
class
IntThingTester : public ThingTester {
public:
    IntThingTester() = default;
    void do_test() {
        std::cout << std::to_string(12 * 12) << '\n';
    }
};
ThingTester* factory() {
    auto t = new IntThingTester();
    return dynamic_cast<ThingTester*>(t);
}

#elif TEST_STRINGS
class
StringThingTester: public ThingTester {
public:
    StringThingTester() = default;
    void do_test() {
        std::string s {"I put my thing down, flip it, and reverse it."};
        std::reverse(s.begin(), s.end());
        std::cout << s << '\n';
    }
};
ThingTester* factory() {
	auto t = new StringThingTester();
	return dynamic_cast<ThingTester*>(t);
}
#endif
// main.cpp
//
#include "testers.h"

int main() {
	ThingTester* the_thing = factory();
	the_thing->do_test();
	delete the_thing;
}
#
# CMakeLists.txt
#
cmake_minimum_required(VERSION 3.15)

project(testers VERSION 0.1 LANGUAGES CXX)

set(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)

add_library(main_obj OBJECT main.cpp)
set_target_properties(main_obj PROPERTIES
  CXX_STANDARD 17
  POSITION_INDEPENDENT_CODE TRUE
)
target_compile_options(main_obj PUBLIC "-Wall" "-Wpedantic")

add_executable(int_test testers.cpp)
target_compile_definitions(int_test PRIVATE TEST_INTS=1)
target_link_libraries(int_test PRIVATE $<TARGET_OBJECTS:main_obj>)

add_executable(string_test testers.cpp)
target_compile_definitions(string_test PRIVATE TEST_STRINGS=1)
target_link_libraries(string_test PRIVATE $<TARGET_OBJECTS:main_obj>)

# After configuring the build dir...
$ cmake -B build -S .

# You have two executable targets you can build, that share main.cpp.o
$ cmake --build build
[ 20%] Building CXX object CMakeFiles/main_obj.dir/main.cpp.o
[ 20%] Built target main_obj
[ 40%] Building CXX object CMakeFiles/int_test.dir/testers.cpp.o
[ 60%] Linking CXX executable int_test
[ 60%] Built target int_test
[ 80%] Building CXX object CMakeFiles/string_test.dir/testers.cpp.o
[100%] Linking CXX executable string_test
[100%] Built target string_test

$ ./build/int_test
144
$ ./build/string_test
.ti esrever dna ,ti pilf ,nwod gniht ym tup I

In fact, if you were to split the testers.cpp contents up into separate files, I’m preeeety sure you could get away without even the preprocessor conditionals / target_compile_definitions(). Which subclass got used would simply be a question of which source file you compiled into the executable.

Hi @ferdnyc,

Thank you for the idea. However, I see the following problems:

  1. Could there be a performance impact? If the compiler does not know the derived class, then it cannot inline the test functions inside main. The test functions are called repeatedly inside a loop, and some work is repeated between them sometimes, which the compiler should be able to notice if the code for both is “written consecutively”. Since afterwards I need to run the files over a lot of traces, I would like it to be as fast as possible. Though it is true that maybe this can be solved thanks to link time optimizations.
  2. How does this method escalate when there are multiple executables that can be built from the same derived classes. For example, if I want to add an executable that can be compiled using two branch predictors and compares them, I then need to add another factory function, and include a definition for it in the source file of each branch predictor.

If possible, I would also prefer to leave untouched the source files for the predictors. That is, not including the factory function there.

Is it possible to have a file that just includes the factory function “template” and that it is used to generate a file containing the function for each executable and that then gets compiled and linked? And that this file is generated inside the build directory, in order to not clutter the main directory.
(I know about configure_file but I do not think it serves this purpose.)

Thanks in advance.