For some time I’ve wanted a different front-end for the CMake ecosystem, one that would make development much more convenient for me. I am opening a discussion here about the possible design. I use cm as the tool name but am not wedded to any details such as the names of things or implementation language.
Here are some things I have in mind:
There is a user preferences file, e.g. ~/.cm-config; lots of things customizable there including default generator and build type.
For all cm commands, there’s a way to pass additional arguments on to the underlying tools.
All commands identify the project to build by looking upward starting from the current directory for a CMakeLists.txt file with a top-level project declaration.
A standard out-of-source build directory is chosen for each top-level project; maybe there’s a ~/cm-builds directory with subdirectories for different source project locations.
cm configure invokes CMake in some standard way.
cm build from anywhere in the source project invokes cm configure first if it hasn’t been done yet, then invokes cmake --build.
cm test from anywhere in the source project invokes cm build first if it hasn’t been done yet, then invokes ctest.
Passing --rebuild to any of these commands blows away the entire build directory and repeats the commands as they were originally issued.
I’d prefer not to add anything like this upstream yet, though I’d appreciate the name “cm” not being taken by a third-party wrapper so we can reconsider this in the future.
I really hope that people will read past the implication that the idea was already discussed in these forums. That thread floats the idea that it should be possible to create post-hoc cmake subcommands like cmake test, presumably by putting an executable called cmake-test in your PATH. As far as I can tell, it proposes no semantics and has almost no overlap with what I’m proposing.
I appreciate the desire not to have names that might be useful to you in the future “taken” by other projects, but from that alone it’s what kinds of names would be acceptable to you. Can you offer any guidance?
If you want to create your own wrapper/launcher to offer a different workflow, I request that it uses a name that doesn’t look like it is part of cmake upstream. It doesn’t even have to have “cm” in it. That would also leave room for the tool to wrap other build systems to provide a matching workflow.
So, a tool with configure/build/test commands that invokes CMake configure/build/test commands? I’m not sure I fully understand the value proposition beyond what current CMake command line usage and CMake presets provide.
Of course I know the name doesn’t have to have any particular string it. Unfortunately, only you know what names look like to you like they are part of cmake upstream. I’d never have guessed you thought cm was one of those; cmake doesn’t have precedent for any unpronounceable-other-than-by-spelling names as far as I know. So please, a little specific guidance about what you’d like me to avoid would be helpful.
As for wrapping other build systems, that’s not a generalization I am interested in. The modern build systems I know already support most of these behaviors OOTB.
Lets me configure, build, and test with a single command: cm test (or whatever).
Lets me do it from anywhere in the project source tree.
Lets me set up habitual defaults that apply to all projects unless I specify otherwise.
Picks a build directory for me unless I specify otherwise.
Doesn’t make me translate my commands and build options to JSON.
Lets me completely reproduce the last configure step while discarding the existing build directory. Searching through terminal history is no fun, and often I don’t have one for the project in question.
Given this functionality, for moderately-sized projects the fastest way for me to get started would be checkout and cm test from the source directory. If something goes wrong, I could make adjustments. Getting started with cmake on any given project is a lot more fiddly than that today.
These things may not matter to you, but they do to me. One of my colleagues said:
Such a CMake wrapper sounds very useful. I’d immediately start using such a thing.
…so I thought maybe other people around here would care. FWIW, I don’t see the appeal of presets and I don’t like using them, but my person I’m quoting has the opposite view, which goes to show that the value of this tool is really orthogonal to what they provide.
I agree this sort of tool is interesting to support.
For the audience, if not Dave specifically, note that most engineers in my organization use the CMake Tools VS Code plugin, which checks pretty much all the feature boxes that Dave listed. It is possible Dave isn’t a fan of VS Code or IDEs in general, which is fine. That’s why I think a CLI tool that supports basically the same feature set would be worthwhile. I also expect such a workflow tool would become popular to use in CI configurations to keep CI configs in each project as terse as possible. Internally at my organization we have such a tool and it’s fairly popular (though less popular than VS Code).
I should also note that it’s my experience that the number one biggest headaches while building in the C/C++/Fortran/ELF/etc. ecosystem is actually user-friendly dependency resolution. When I consider other build tools tools like gradle, maven, cargo, gobuild, etc., a step to fetch and sometimes build dependencies is typically included in the development workflow. I bring this up mostly because Dave didn’t list this as a requirement or non-requirement from his perspective.
I also bring it up because whether and how to support dependency fetching is one of the biggest decisions to make when scoping a build tool. It’s also relevant because part of the reason my organization can’t really release its internal CMake-based CLI tools is because our dependency resolution logic is not something one could just install and use outside of our network. Given there aren’t adopted dependency resolution standards yet, my instinct is to explicitly scope out dependency resolution initially, instead investing in technologies like CPS to provide “your dependency isn’t installed” diagnostics to users.
Aside on Presets: It’s true that some of Dave’s feature list is covered by workflow presets, especially the feature involving a one command end-to-end workflow. Though we still have this long-open issue about how presets have a difficult time modelling all the interesting workflows for presets because they are fundamentally an enumeration of interesting combinations of settings. But modern C++ projects have O(m*n*p*q*r*s) combinations of interesting build flavors to support. Just to illustrate all the possible choices I regularly make, I might want to build with: “debug”, address sanitizer, undefined behavior sanitizer, integration tests enabled, strict warnings checks, using a specific CMake toolchain file, and Ninja. Or maybe I want all that but using Unix Makefiles? Preset files are effective for defining specific settings like that for a small number of projects, but they are not effective in supporting more than a handful of preset files. The burden to maintain preset files grows O(n) with the number of presets, and the number of interesting build configurations is basically unbounded.
Note that the Beman Project would benefit from a tool in this space. I’ve been putting off inventing this sort of tool for a few reasons, including:
It’s not a primary goal of the Beman Project to solve ecosystem-wide developer experience issues
Making sure a new tool is installed on all possible environment is also a challenge
I have a lot of other things going on
The Beman Project exemplar is intended to be a trivial project that any standards library proposal can derive from (it’s a GitHub project template). But I still see a lot of room to develop. For instance, I don’t see how to make the presets file exhaustive given the goal to support all of MSVC, Apple clang, GCC, other clangs, sanitizers, etc., etc. I also don’t like how much CMake-specific logic we need to put in the CI configuration, especially compared to a toolname test command that could be invented and require little-to-no configuration per repo in principle.
I would also scope in (and have!) the following to a workflow tool:
I had a some local drafts with the proposed in my own discourse thread, and I am planning to publish it in near time in my fork here: https://gitlab.kitware.com/leha-bot/cmake
I’ll be very glad if we can cooperate on this feature together
Having a place to store user defined defaults which are then shared across projects would really make my work easier.
Presets are a way for projects to specify project-specific defaults (also the CMakeUserPresets.json file is project-specific, as it has to be copied into the project’s root directory, where it then diverges from other copies), which is something that I strongly discourage. There should be no project-specific defaults, ever! Developers switching from one project to another should either find the exact same defaults defined by the toolchain, or they should be able to reuse their user-specific configuration. Everything else creates more problems than it solves.
Another thing that I dislike as much as CMake presets are Github workflows. They too are copied into the root directory of each project, and then diverge there. It would be great if a simpler frontend could reduce the need for that too.
Those two preferences are mutually exclusive, in a way. If we want the name cm reserved, it should be reserved upstream rather soonish.
Interesting, but if I may ask what do you mean by “workflow tool?” Is what I’m proposing here a workflow tool, and if so what makes it one?
Also, what would it mean to “scope in” the three things you mention above? The tool I’ve proposing, as I’ve proposed it, is only trying to provide a better way to drive cmake and ctest. I don’t see any advantage to broadening its scope beyond that, when other problems can be solved by other tools, but maybe I’m missing something.
Not for the tool I’m proposing, since @brad.king wants it very clearly to not appear to be part of the upstream CMake ecosystem.
Hi Daniel!
Yes, the problems with GH workflows are legion, among them the fact that as you point out it’s very hard to share things across projects. That said, I don’t see how a “front end tool” for anything can help with that, nor do I see how that problem is related to CMake. What am I missing?
Reading your usage proposals, I think presets would get you most of the way there.
The latest schema version has workflow settings; I haven’t used them but it seems like they’d address the “configure, if necessary, then build” requirement. Also the build & test flow.
It also has include directives. If you want global configuration for all your CMake projects then perhaps this could be leveraged with
Presets have configure defaults for the generator, build dir (binaryDir), and invoking CMake in some standard way. I too have often wanted the “nuke the build tree and reconfigure with the same command as last time” option. That is a missing feature I’d say. If you added/changed anything on the command line when you --configure’d that is not captured anywhere.
Other items on your wishlist might be better handled as shell configuration. For example, being able to dispatch a CMake command from anywhere in the repo tree might involve a shell function to resolve the Git root (git rev-parse --show-toplevel), and cd’ing there, before calling the cmake command.
All the items in your list are perfectly reasonable. In my perspective a good deal of it is already possible if you’re willing to work with presets and then smooth the edges with some shell support.
What is a workflow setting and where can I learn more about it?
Then the workflow would only be available in projects that use this pattern? That’s not what I’m after.
Presets have configure defaults for the generator, build dir (binaryDir ), and invoking CMake in some standard way. I too have often wanted the “nuke the build tree and reconfigure with the same command as last time” option. That is a missing feature I’d say.
I could be mistaken, but given the reception my inquiries about it have received here in the forums, my impression is that it’s viewed as inappropriate for CMake and thus is going to stay missing.
Other items on your wishlist might be better handled as shell configuration.
No thanks; that makes getting this functionality a programming and personal configuration exercise for every developer who wants it.
In my perspective a good deal of it is already possible if you’re willing to work with presets and then smooth the edges with some shell support.
Sure, I could cobble something together, but it would be a one-off, undocumented, unmaintained, and probably not easily used by anyone else. If I keep smoothing edges and adding shell support I can see that this ultimately ends up as a front-end tool that works on all platforms, independent of version control system, that can be installed as a single entity, has first-class documentation, good diagnostics for incorrect usage, etc.
Environment variables are the last thing I’d want to depend on. I might even be inclined to have my front-end tool optionally clear environment variables before invoking tools, to eliminate effects that are hard to track down.
Sure, I could cobble something together, but it would be a one-off, undocumented, unmaintained, and probably not easily used by anyone else. If I keep smoothing edges and adding shell support I can see that this ultimately ends up as a front-end tool that works on all platforms, independent of version control system, that can be installed as a single entity, has first-class documentation, good diagnostics for incorrect usage, etc.
Nobody’s stopping you. The closest thing to that has been a maintainer request to choose some non-branded name.
Environment variables are the last thing I’d want to depend on. I might even be inclined to have my front-end tool optionally clear environment variables before invoking tools, to eliminate effects that are hard to track down.
Environment variables are a configuration point. If the user does it at a system level or user level, then it’s an analogous mechanism to the sort of config file idea you’re proposing.
I could be mistaken, but given the reception my inquiries about it have received here in the forums, my impression is that it’s viewed as inappropriate for CMake and thus is going to stay missing.
I’m not familiar with whatever discussion you’re referring to, but there’s at least one “CMake frontend” that has this, and I mentioned it in my previous comment.
What is a workflow setting and where can I learn more about it?
There is a user preferences file, e.g. ~/.cm-config; lots of things customizable there including default generator and build type.
As I said, you can do this with environment variables, and I’d rather use a CMake-provided way than a wrapper tool, assuming config of wrapper tool only applies to the wrapper tool. CMake-provided environment variables are interpreted by CMake, so any frontend (basically every IDE or IDE plugin invoking CMake, and ccmake, and cmake-gui) that invokes CMake will pick up on this unless it forcefully passes -D to take priority, or the user configures at the IDE/plugin level to overwrite the environment variables that would otherwise be inherited from system/user-level config.
For all cm commands, there’s a way to pass additional arguments on to the underlying tools.
cmake --build already has [-- <build-tool-options>]
All commands identify the project to build by looking upward starting from the current directory for a CMakeLists.txt file with a top-level project declaration.
Say I have a :/external/somelib git submodule, or fetched via FetchContent. If I cd into there and run this command, it’s not going to do what I want, and there’s no “in-channel” backreference to the top level project. Whereas CMake Tools knows what my top-level projects are because it can infer them, or I tell it.
A standard out-of-source build directory is chosen for each top-level project; maybe there’s a ~/cm-builds directory with subdirectories for different source project locations.
And how is naming going to be automated? Does the user get control? How does the user communicate the parameters / format for generated names? Consider that binaryDir already supports macros.
cm configure invokes CMake in some standard way.
I don’t see the value in this given that I’m not seeing value for myself as a whole here.
cm build from anywhere in the source project invokes cm configure first if it hasn’t been done yet, then invokes cmake --build.
In the case where the buildsystem has already been generated and configuration has changed, cmake --build already handles this. For the case where a buildsystem hasn’t been generated, CMake Tools + CMake Presets already does this, and I’ll bet other IDEs do something similar with presets configured.
cm test from anywhere in the source project invokes cm build first if it hasn’t been done yet, then invokes ctest.
Note: CMake Tools does this. I don’t see myself using an IDE other than VS Code for personal use, and if I wanted keyboard shortcuts in my shell, given that bash is available for every major platform, I’d just write a binding in inputrc.
The feature in question was being referred to as a “missing feature,” which carries the implication that it really should go into CMake, as opposed to some frontend tool, and that implication was what I was responding to.
Regarding environment variables, yes, they are a configuration point. They’re also liable to be different in my terminal than in a subprocess of my IDE, and it’s easy to make a change in a terminal session and forget that it’s only local, or to forget to set the variable wherever it will take effect more permanently—oh and by the way, the place to make that permanent change depends not only on your OS but on the particular shell you choose to use on that OS. Configuration files are not subject to the same kinds of pitfalls.
Note: CMake Tools does this. I don’t see myself using an IDE other than VS Code…
It’s clear that what I’m proposing isn’t for you, so I’m not responding to the how-does-it-work challenges you’ve posed for your benefit. For anyone else,
I’m fine with builds from within a submodule or downloaded dependency that contains a cmake project only building that project. Those are rare scenarios for which I don’t mind changing directory. The point of this tool is to make most things I do on a daily basis convenient.
Initial user control over out-of-source build directory naming will be to specify where a root under which all build directories are collected. I would not complicate that by adding further configuration unless it was shown to be insufficient for a lot of users. Details of how subdirectories are chosen are fungible; for platforms with path length limits we might hash the path to the source directory.
A frequent obstacle newcomers have with CMake projects is the complexity of build instructions. Presets help a lot, but even something like cmake --workflow --preset gcc is difficult (What if I’m using a different compiler? What if I also need to set a CMake cache variable? Why doesn’t this preset work in a different CMake project? What will that command do? [configure? build? test? package?])
The Rust folks have spent considerable effort on ergonomics to lower barriers to entry and ended up with something similar to @dabrahams’s proposal (cargo build, cargo test, etc.). The convenience of Rust’s build system has been an important contributor to Rust’s popularity.
I don’t know if it looks exactly like @dabrahams’s proposal, but think there’s a good opportunity here to improve CMake’s user experience, especially for new users.
Any attempt to make CMake invocations “easier” will either need to be quite opinionated and hide extra details from the user, or if all the extra settings are allowed, then the tool will end up basically just replicating CMake’s original command line interface (or will look different but have a similar complexity).
If the reasoning is that CMake is complicated to learn, I don’t really see why making users learn yet another wrapper tool makes things simpler. If anything, it makes the ecosystem more complex to understand: a wrapper tool to invoke a build system generator to create project files… not easy for a newbie to wrap their head around!
I don’t mean to be negative, but I really struggle to find much value in this proposal, I think that time should be invested in improving CMake docs & tutorials and the presets features, and users should still be using the official CMake CLI, whether that’s individual configure/build/test commands or workflow presets.
