Ninja multi-config is overly complex

Development
1

After giving the new Ninja Multi-Config generator a spin and trying to wrap my head around its associated documentation, I’m concerned that things are currently more confusing than they should be and that the defaults are not consistent with the behavior of other multi-config generators. There seems to be too much focus on cross-config support at the expense of the much more likely scenario where cross-config support isn’t needed. I think we can and should address this before the 3.17 release.

Default configuration (part 1)

My first experience was typing ninja with no command line arguments and being met with an error that build.ninja was missing. I later worked out that I had to set the CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG variable if I wanted that to work. For other multi-config generators like Xcode and Visual Studio, you get the Debug config by default if you don’t specify a configuration. Indeed, if you do cmake --build . without specifying --config, you also get the Debug configuration with the Ninja Multi-Config generator, so it seems inconsistent that invoking the ninja build tool directly doesn’t work out-of-the-box.

After wrapping my head around the relevant variables, it seems to me that we shouldn’t need the CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG variable at all. In my opinion, we should always have a default config if one isn’t specified and we can use the first one listed in the CMAKE_CONFIGURATION_TYPES variable for that choice. This is what the Xcode generator does. The Visual Studio generator always builds the Debug config by default if you invoke msbuild from the command line without specifying a configuration, but the Xcode generator’s behavior is more intuitive (to me). Edit: Invoking xcodebuild directly will build the first config in CMAKE_CONFIGURATION_TYPES, but invoking cmake --build apparently always builds the Debug config regardless of the generator type (see this comment by @kyle.edwards) .

Proposed change: Get rid of CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG and use the first config listed in CMAKE_CONFIGURATION_TYPES instead. This also means there will always be a default configuration and there will always be a build.ninja file.

Default configurations (part 2)

The CMAKE_NMC_DEFAULT_CONFIGS variable is currently used to define the set of configs that should be built if no configuration is specified when providing a target. I’m wondering if we really should be providing this. Shouldn’t that be the job of whatever is driving the build step? You can loop over the configs if driving it from a script. For an end user, how often is anyone realistically going to want to build more than one config in a non-scripted scenario? I’m sure there are valid use cases for this, but I have reservations whether the complexity and potential confusion this feature introduces is worth it.

Proposed change: Remove the support for building multiple configs with one command. This will mean we can remove CMAKE_NMC_DEFAULT_CONFIGS.

Variable naming

The use of _NMC in the variable names is not particularly user-friendly – new users probably won’t know what it stands for if they see the variable without any context (e.g. in a project’s scripts, etc.). I’m wondering if we need to specify anything generator-specific in these variable names at all. We could leave it out and then document that those variables only apply to the Ninja Multi-Config generator. Other generators could just ignore them, or if they gain equivalent functionality in the future, then we already have the api ready to support it for them.

Proposed change: Remove the _NMC_ from any variable names and document them as only being supported by the Ninja Multi-Config generator.

Documentation flow

The main Ninja Multi-Config documentation page could be reworked to give the reader a gentler introduction to the features. I would recommend moving things related to cross-configs to a dedicated section at the end of the page. Let the start of the page be about multi-config support that looks and behaves like the other multi-config generators. That will encourage more people to try it out, especially if they are already familiar with Xcode or Visual Studio. We already acknowledge that most people probably won’t be interested in cross-config support, so better to leave it to the end where those not interested in it can just skip it and be happy with the basic features.

Proposed change: Rework the Ninja Multi-Config generator’s main doc page to move the cross-config contents to its own section at the end.

Documentation of individual variables

The separate pages that document each variable contain minimal content and instead refer back to the main Ninja Multi-Config page for details. This seems backwards. These separate pages are supposed to be the definitive reference for these variables, so they should be the most detailed and contain all information relevant to them.

Proposed change: Move the detailed descriptions into the individual variable pages and use the main Ninja Multi-Config generator page to focus on examples and how the variables work together. The main generator page could become more scenario-based rather than feature/variable-based.

3 Likes
2

Thanks Craig. I’ll go through your points one by one, explain the reasoning behind the decisions, and discuss ways to address each of the issues.

Indeed, perhaps we could set CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG to Debug by default, so that build.ninja is always available.

We discussed the idea of doing something like this, but I rejected it. I’ve always been of the mindset that “explicit is better than implicit” (taken directly from the Zen of Python, and the Rust philosophy seems to agree with me too.) Using a separate variable explicitly states your intentions, whereas taking the first element of an only semi-related list is more implicit. I also conceptualize CMAKE_CONFIGURATION_TYPES as an unordered set rather than a list. I thought that the Xcode generator used Debug by default rather than the first element of CMAKE_CONFIGURATION_TYPES, but I could be wrong.

This feature was requested by the Qt Company, who provided the funding for the Ninja Multi-Config generator. Even if some people don’t use it (and there is a sensible default if it’s not specified), others have a business requirement for it, so we shouldn’t get rid of it.

The CMAKE_NMC_ variable names were selected in the same spirit as the CMAKE_VS_ and CMAKE_XCODE_ variables. They were initially CMAKE_NINJA_MULTI_, but this made already lengthy variable names way too long. @brad.king and @ben.boeckel and I had an extensive discussion about the naming. There’s a reason why “naming things” is listed as the second hard problem in computer science :slight_smile: . Since these variables are potentially more likely to be used in another generator later on (multi-config Makefiles perhaps?) I’d be willing to consider removing the NMC_ portion. I’ll have to discuss it with my colleagues.

It was kind of my intention to do this anyway, and I tried to put it to the end as much as I could. The problem is that the cross-config documentation is rather lengthy, so it gives the impression of being the main focus of the page. I’d be willing to look at ways to segregate it out into its own section.

I kind of did this deliberately. One of the things I find somewhat frustrating about the documentation is that there are lots of individual pages to learn about the many features (or components of a single feature), but not much in the way of a central outline to give a holistic view of how to use it. Perhaps this role is better suited for a tutorial (or a book? :wink: ). We’ll have to figure out a way to move the information into the variable documentation without losing information or readability.

3

@craig.scott thanks for reviewing this. We do need to keep all the features, but we can simplify the usability.

I like the suggestion to drop the _NMC part of the variable names. That way the same settings can be used for other future multi-config generators. For now we document them as only available for this generator. We should add errors to the other generators if the variables are set on a generator that does not support them (to leave room for enabling them in the future).

This generator is primarily meant for build scripts that want to prepare a distribution, and for those the explicit settings like CMAKE_NMC_DEFAULT_BUILD_FILE_CONFIG (now named CMAKE_DEFAULT_BUILD_FILE_CONFIG) make sense. However, I’m sure people will start using it for local development too, e.g. to build Debug most of the time for incremental development but also build Release for testing. For such cases it is indeed a lot to type out an extra -DCMAKE_DEFAULT_BUILD_FILE_CONFIG=Debug argument. Since we already make cmake --build . implicitly default to the Debug configuration for other multi-config generators when no --config ... argument is given, I think it is reasonable to implicitly select the first entry of CMAKE_CONFIGURATION_TYPES (which is Debug by default) as the build.ninja configuration. We should keep the explicit CMAKE_DEFAULT_BUILD_FILE_CONFIG setting for scripts to use when they do not also set CMAKE_CONFIGURATION_TYPES but otherwise can use the latter’s order for the default.

4

That’s a good compromise, I like that. It gives better defaults but keeps the explicit control for those who want a dedicated option for it. Based on my experiments, I suspect we can also make this work for Xcode too, since it seems to already be using the first CMAKE_CONFIGURATION_TYPES entry as the default for xcodebuild. We could probably rename the option to something like CMAKE_DEFAULT_BUILD_CONFIG, or perhaps for more consistency with CMAKE_BUILD_TYPE and CMAKE_CONFIGURATION_TYPES, we could call it CMAKE_DEFAULT_BUILD_TYPE instead? Something like that would have really helped with understanding the docs on my first read.

5

And taking it a step further, CMAKE_DEFAULT_BUILD_TYPE could be what cmake --build . uses instead of always defaulting to Debug.

6

Actually CMAKE{,_NMC}_DEFAULT_BUILD_FILE_CONFIG used to be called CMAKE_NINJA_MULTI_DEFAULT_BUILD_TYPE but we changed it to clarify that it selects which build-<Config>.ninja is used as build.ninja rather than what configuration’s artifacts to build. That distinction is very specific to this generator and its cross-config settings. I think that use case is advanced enough that anyone who understands it will be able to figure out what the name CMAKE_DEFAULT_BUILD_TYPE means. In the common/default non-cross-config case it happens to also mean the artifact configuration anyway.

Therefore I think we can go ahead and also rename CMAKE{,_NMC}_DEFAULT_BUILD_FILE_CONFIG to CMAKE_DEFAULT_BUILD_TYPE. Using it for cmake --build . makes sense too, though note for that we can only do it if it is literally in CMakeCache.txt.

7

Keep in mind that the CMake docs are the authoritative reference manual for CMake. In my view, that is its primary purpose, although the addition of guides more recently expands on that. As a reference, the documentation for each specific thing should give the details about that thing. It’s okay to cross-reference to other related entities, but if I want to know what a specific variable does, I should be able to go to that variable’s docs, not have to search through a different page which describes a number of different aspects of a bigger picture. Having that longer, more complete description is also useful, but that should focus more on how things work together and the individual variable pages should give all the precise details about that thing. Yes there will be some repetition, but not usually in a bad way. This is a common pattern (see the page for the Visual Studio 16 2019 generator to get an idea of what I mean). The module docs are less consistent, but they are an everything-on-one-page situation anyway.

1 Like
8

I agree there should be more detail in the individual pages even if it results in some repetition. It is reference documentation after all. However, let’s focus on finalizing the set of variables and their names first. We can polish the documentation after that.

9

At the risk of taking this too far, I wonder if we should also consider using CMAKE_DEFAULT_BUILD_TYPE to provide the initial value for CMAKE_BUILD_TYPE for single-config generators (if not set already)? This could be an opportunity to stop making an empty build type the default on single config generators and instead have it be Debug (which is what people typically assume it actually is). We could do this under the control of a policy to preserve backward compatibility.

10

Let’s defer further semantics of CMAKE_DEFAULT_BUILD_TYPE to future work. For now it can be an error for single-config generators so that we can expand the behavior later.