How to change toolchain without breaking developer workflows?

Here is the short of it.

I have a toolchain. The toochain specifies these variables:

CMAKE_RC_COMPILER=.../rc.exe
CMAKE_CXX_COMPILER= .../cl.exe
CMAKE_C_COMPILER= .../cl.exe
etc...

I update the toolchain to use a newer compiler/sdk etc.
This didn’t seem to be a problem with Visual Studio as it succeeded just fine. (Which is a problem in itself in my opinion)
However, this fails on Ninja with the following error message.

-- Configuring done
You have changed variables that require your cache to be deleted.
Configure will be re-run and you may have to reset some variables.
The following variables have changed:

Granted I don’t have to delete the build. I just need to rerun cmake. But is there anything I can do to avoid this problem? Because it’s pretty jaring to developers on our current team to have this error.

If you change the toolchain, you should start with a fresh build. There are too many things that assume the toolchain doesn’t change and while you may be able to find workarounds which appear to work, I recommend you always use a fresh build tree for a different toolchain. This same logic also applies if you update the existing toolchain in-place (e.g. you update to a newer version of GCC on Linux, a newer version of Xcode on macOS, etc.). CMake queries compiler capabilities and caches the results. If you change the toolchain in a way that CMake can’t catch, then you end up with stale cached capabilities being used for the new/updated toolchain. Please don’t do that. :wink:

If you want to switch back and forth between toolchains, set up two different build trees and then you can jump between them as needed. You can have both associated with the same source tree. I do this frequently myself.

Is there a way to create a fresh build for the user? So that the onus is on the cmake writer/maintainer.

Because ideally I don’t want people to have to manually delete their build directories themselves.

It would be really nice if this was possible for example

if (${BREAKING_TOOLCHAIN_ISSUE)
   // Delete cmake cache/build
   // Call configure/generate for the user with the command line arguments the user passed in
   restart_cmake()
endif()

The user should be in control of wiping their own build directory or CMake cache. CMake can do its part by halting if it detects a change. The user action at that point is trivial for the user to do. Perhaps we can improve the warning/error message to make that action clearer, but I would not be in favour of CMake discarding a user’s CMake cache on change of toolchain. Accidentally using a different toolchain is relatively easy to do, especially with the way some IDEs work and you are jumping between command line and IDE for the same build. It happens to me quite regularly. Having to re-create all my cache settings in such accidental cases would be a big inconvenience.

In a CI pipeline, there are multiple things that could make CMake fail. Users could have committed a bogus version of the CMake project, or they could have updated the toolchain and the old cache stands in the way. How would one make the CI agents resilient to successive compiles on different branches using different versions of the toolchain? Could CMake fail with a specific error code?

Ordinarily, you’d try to set up CI such that every build is a fresh build. Things like sccache or ccache are usually very effective at reducing the build times for that. But it is also not that unusual to see CI use incremental builds too. Some projects have characteristics that mean making every build a fresh build isn’t feasible. In that case, you could look at having the CI script delete the CMakeCache.txt file at the start. That would make it a fresh build from CMake’s perspective, but the build step may still re-use some of the existing build results. YMMV for this, as removing the CMakeCache.txt file may trigger many of the build outputs needing to be rebuilt. It depends on your project. Still, using sccache or ccache as well may still be enough to get this second strategy to work (I’ve seen exactly this in some companies and it was reasonably effective).

My concern is less CI. And more the users of CMake. Because as you mentioned clean builds + sccache or ccache is a good way to go.

Could CMake fail with a specific error code?

How does CMake treat toolchain updates?
Since you aren’t supposed to change it, what does the error message look like?

You’re moving the goalposts. :wink: I don’t know off hand what the error message is if you switch to a different compiler. From memory, when CMake detects a different compiler to the one used in the last run, it will issue a warning and re-test the compiler as though it was a new run. But I think that detection is fairly limited. I wouldn’t expect it to trigger if you merely update the same compiler in-place. In that case, CMake would re-use its past detection results (which may now be wrong). For the actual build, I don’t know if each generator includes the compiler executable in its dependencies for compiling each file, but maybe they do. You’d need to check yourself.