Using Interface as specs with subdirectories

Code
1

In the interest of clarity and capturing requirements clearly, I’m attempting to use INTERFACE targets as a mechanism to group compiler flags, definitions, and linker options. An example is like so:

add_library(my_specs INTERFACE)
target_compile_definitions(my_specs INTERFACE MY_DEF=FOO)
target_compile_options(my_specs INTERFACE -mfoo=bar)
target_link_options(my_specs INTERFACE -motherfoo=otherbar)

In a project, I’d use it like so:

add_library(my_lib STATIC ... )
target_link_libraries(my_lib PRIVATE my_specs)

add_executable(my_elf ...)
target_link_libraries(my_elf PRIVATE my_lib my_specs)

And this generally works, the compiler inserts the definitions and options for sources in my_lib and my_elf, and the linker applies the linker option -motherfoo=otherbar, all good.

But if my_lib comes from a subdirectory or elsewhere:

add_subdirectory(path/to/my_lib)

And it contains some dependencies:

# path/to/my_lib/CMakeLists.txt
find_package(ExternalPackage REQUIRED)

add_subdirectory(child_dependency)

add_library(my_lib STATIC ... )
target_link_libraries(my_lib PRIVATE child_dependency lib_provided_by_external_package)

Then child_dependency and lib_provided_by_external_package are NOT built with the compiler flags and options from my_specs, and the linker options are not used when linking them, even though they are used for my_lib itself.

So when the whole thing is assembled at the executable stage, there can be complete binary incompatibility (when building for, e.g. ARM and one library is compiled with the hard float ABI and the other is not) and the whole link fails (not to mention that even if it could succeed you would have some code using the wrong ABI for float).

Without knowing all of the dependencies and injecting the specs into each of them at the top-level CMakeLists.txt file, and assuming the library is built for various specs at various times and thus the specs cannot be hardcoded into it, what is “the CMake way” to solve this problem and ensure the spec requirements are carried down to the lowest dependency in the library?

2

It’s pretty fundamental to build systems that components cannot be influenced by what uses them, only by what they consume (for include directories, compile definitions, link line arguments, etc.). So the fact that child_dependency is used at some place where my_specs is also used is just not something that’s going to happen. You’ll have to change child_dependency’s recipe itself to take my_specs into account. Same with external_dependency targets. CMake doesn’t even know how to build those, so making them rebuild with some flags based on where they’re used isn’t possible.

3

Ok, but how is this any different from using add_compile_options() to globally change things? That affects everything from that point on down - which strikes me as a rather imperative approach rather than the CMake prescriptive approach.

I mean, if that’s the only way to do this, then I’ll do it (and probably look for an alternative to CMake), but if I build a library with specific hardware requirements, it completely makes sense that its dependencies be built with those same requirements - that’s sort of the point of a library: to provide a factored capability for the consumer of the library - which implies a coupling in build settings among other things. I don’t really see how it makes sense that the child dependencies shouldn’t see these same requirements - in fact it’s a guaranteed way to build incompatible libraries out of the same dependency chain.

4

I think you mean “declarative”, but yes. Its addition was probably to match the existence of include_directories, compile_definitions, etc. I wouldn’t mind seeing these go away in the future, but there’s a lot of code out there that uses them.

I don’t think any build system supports “give me things to compile on based on where and how I’m used”. It’s completely backwards and generally unsolvable. See this message on the gn-dev list where something very similar was asked:

A target can never modify its dependencies. Otherwise, that creates a situation where that other target has an ambiguous definition based on who is depending on it. In addition to forcing a target to have multiple definitions, this would also create unsolvable build graphs if the same target is referred as a dependency through multiple paths, each with their own flags.

How is CMake to know which requirements are “platform specific” versus “target specific”? There’s no syntax for that today.

Other things to consider:

  • What if two usages end up with conflicting requirements on a library? Does it split into two or just error with “undecideable”?

This is where you use a toolchain file to describe your target platform. That way everything gets that baseline requirement. You’re still left with external dependencies needing to use the same toolchain setup to agree, but recompiling external dependencies is just never going to be a thing any C or C++ build system is going to reliably support.

5

Hm, there might be some crossed wires here. Literally every part of this code is compiled as part of this build chain, since it’s being cross-compiled for a remote target. So we recompile our external dependencies all the time. This is entirely normal in the embedded world.

That can be fixed, and IMO should be.

Well, I suppose we disagree on this. In many cases it is entirely solvable - especially if I’m the one recompiling all of the dependencies as static libraries to be built into the executable I define at the top level. This is not an uncommon case, although it might be uncommon for most CMake users.

6

Toolchains are just that: toolchains, not board descriptions. A toolchain file is the wrong place to put specific board details that share the same toolchain.

For instance, if I’m building on the ARM platform, I’ll use an ARM toolchain which describes my arm-none-eabi-* compiler binaries and locations, and basic other definitions.

But I’ll build for a whole variety of target chips with that toolchain. Right now, I set globals for each chip, which is difficult to maintain as threading through which particular set of globals got defined by which path through the build code turns CMake into Makefiles.

It’d be much cleaner if I encapsulate the board-specific build requirements into an interface library, and then just choose to build the top-level executable against the appropriately selected interface library, as per my example. Since the executable is packaged separately from the bulk of the core code, which are all static dependent libraries in a large hierarchical chain, this theoretically should permit the building of everything that executable incorporates statically into one coherent whole.

Since this has been done for embedded systems using Makefiles which incorporate a trickle-down sense of dependency for embedded libraries since nearly forever, I find it difficult to accept that this is not a build tool concern.

7

OK, the mention of the external library is what made me mention that.

It is something probably done other ways in CMake (either via toolchains or other global mechanisms). There are embedded developers on this Discourse instance, but I don’t know how often they end up perusing to help others.

It might be more that CMake’s (and gn’s) models never needed it and so they’re not equipped to handle it. Even so, I don’t think CMake is going to gain support for dependents modifying dependencies anytime soon. Myself, as someone who maintains a complex build system, knowing that the places I need to look are all dependencies of what I’m looking at is far simpler than also having to consider “where is this used” as well.

8

Certainly. I have a working build chain already that I use which uses these mechanisms. I just find it not tremendously less complicated than the crufty Makefiles it replaced.

This is a problem with a lot of modern thinking when it comes to tooling: “I don’t need it so nobody else should”. I understand that in the traditional “everything on the same platform” dynamic-library based environment it is typical to consider libraries and other dependencies as standalone entities - we’ve all built apps for UNIX and are well familiar with the paradigm.

I appreciate your candour in addressing my question and concerns. :beers: Hopefully some of the embedded folks have some ideas I could apply, if they drop by this question too. Otherwise, I’ll drop my attempt to use interface libraries for specs and just stick with global compiler settings for now.

9

CMake has a lot of history and does try to do what it can, but some things have just never come up. And we’re stuck with some of the old decisions no matter how much we’d love to fix them (cf. list representations and the ; separator). I feel like the model that CMake provides doesn’t support dependencies being modified by their consumers (e.g., generator expression evaluation now needs a recursion detector, conflicts need managed, and probably some other things).

I would recommend searching for users that have commented here about ARM board configurations or filed CMake issues about it and pinging them. If some solution does appear, please feel free to submit it as some part of the CMake documentation (guide? examples? not sure where off-hand). Even just writing up an issue pointing to how it was done somewhere so someone else can summarize it would be useful. Most CMake developers aren’t embedded developers, so the finer details of that world aren’t generally understood by us (today).