Looking for general advice on using toolchain files

Usage
1

I’m trying to port some of my code over to using Toolchain files, but I’m running into some difficulties. The documentation is pretty sparse here, and it basically just tells you what toolchain files are, but there’s a lot of info that I feel is missing in order to help people use them effectively. I’ll probably come up with some more questions later, but for starters:

  1. What builtin variables am I allowed to read during processing of a toolchain file? Some variables (like CMAKE_SYSTEM_NAME, CMAKE_<LANG>_COMPILER_ID, etc) are not set yet, while others (like WIN32, CMAKE_GENERATOR, etc) are. I understand why this is for the specific variables I outlined, but the total list of builtins is quite large, and it would be nice if there was a documented list of variables that were known to be available at Toolchain file processing time.

  2. What is the recommended way of setting per-configuration flags (compiler, linker, etc) in a toolchain files? The documentation seems to recommend using CMAKE_<LANG>_FLAGS_<CONFIG>_INIT, but I’m finding this doesn’t work well for me. Because CMAKE_<LANG>_FLAGS_<CONFIG> is initialized from the _INIT version and then stored as a cache variable, a modification to the toolchain file which changes the _INIT variable won’t get picked up until someone deletes their cache. I’m considering workarounds like forcefully unsetting CMAKE_<LANG>_FLAGS_<CONFIG> in the top of my toolchain files, but this seems like a huge hack. I’ve also considered just force setting CMAKE_<LANG>_FLAGS_<CONFIG> directly and bypassing the _INIT versions.
    Note that it is a non goal of our build system to allow the user to specify different flags on the command line. I would even go so far as to say we’re actively hostile to this. We don’t do this at the moment, but just to illustrate what I mean we have considered making a change that if CMAKE_<LANG>_FLAGS_<CONFIG> or the non config version are set prior to project(), we immediately error out with a FATAL_ERROR. I mention this only because it means we are aware that it is considered bad practice or an anti-pattern to prevent the user from customizing their build by passing in their own flags, but we are consciously doing this anyway. I can go into more detail if anyone asks.

  3. What builtin CMake commands am I allowed to use in a toolchain file? Can I use add_compile_options(), add_link_options(), etc? These commands are documented as modifying the COMPILE_FLAGS and LINK_FLAGS directory properties, and it seems weird to modify a directory property from a toolchain file, but I tried and it seems to work. is this a coincidence, or is this a reliable / recommended technique?

  4. What do I do when my toolchain file itself needs configuration variables? The toolchain files themselves appear to be able to see the values I set via -DFOO=BAR on the CMake command line, but these aren’t passed to the try_compile() commands that run during project(). So if my toolchain files depends on those, then all of the try_compile() steps fail. An example of this is that I have a toolchain file specific to using Ninja + MSVC. We set the CMAKE_CXX_COMPILER etc from within the toolchain file to point to a binary with the VS version in the name of the exe. So for VS2017, we have: vs2017-cl.exe, vs2017-link.exe, vs2017-mt.exe, etc. And for VS2019 it’s the same but with a different file name. So I want to allow the user to pass -DMY_VS_VERSION=2017 and then construct the name of the executable based on that value. This causes try_compile() steps to fail since they don’t see the MY_VS_VERSION variable.
    Another example of this could be that I sometimes want to configure my build to use address sanitizer, and sometimes I don’t. This seems like something that should be part of the toolchain file, so the user can pass -DUSE_SANITIZER=address and the toolchain file would append the correct options. This, btw, is another reason why using the _INIT series of variables for compiler flags isn’t great, because you might reconfigure with a different USE_SANITIZER value and now the compiler options change, but the flags that were initially computed from the _INIT variables were cached.

  5. I’ve seen guidance that suggests that anything related to compiler / linker flags probably belongs in a toolchain file. But we have a lot of complicated logic that decides which flags to append, and when. For example, we enable Link Time Code Generation, but only if two specific -D options were passed at the same time, and only with a specific CMAKE_SYSTEM_NAME. This doesn’t seem to fit well into the toolchain file model, and I’m wondering if something like this is better in the CML.

  6. I don’t want any of the default options that CMake uses for CMAKE_CXX_FLAGS_<CONFIG>, I want to build everything myself. The builtin options appear to be coming from this section of code in Modules/Platform/Windows-MSVC.cmake

      string(APPEND CMAKE_${lang}_FLAGS_INIT " ${_PLATFORM_DEFINES}${_PLATFORM_DEFINES_${lang}} /D_WINDOWS${_W3}${_FLAGS_${lang}}")
      string(APPEND CMAKE_${lang}_FLAGS_DEBUG_INIT "${_MDd} /Zi /Ob0 /Od ${_RTC1}")
      string(APPEND CMAKE_${lang}_FLAGS_RELEASE_INIT "${_MD} /O2 /Ob2 /DNDEBUG")
      string(APPEND CMAKE_${lang}_FLAGS_RELWITHDEBINFO_INIT "${_MD} /Zi /O2 /Ob1 /DNDEBUG")
      string(APPEND CMAKE_${lang}_FLAGS_MINSIZEREL_INIT "${_MD} /O1 /Ob1 /DNDEBUG")

But this code runs after my toolchain files. AFAICT the recommended online workarounds of using string(REPLACE) after project() returns seem to be the only thing I can do here, but I’m wondering if anyone has any other suggestions.

  1. We support a very large number of build configurations. Here is a partial list:

Windows Desktop:

  • Generator: Ninja | MSBuild
  • Architecture: x64 | x86
  • Compiler: clang-cl (only with Ninja) | cl

Windows Store:

  • Generator: MSBuild
  • Architecture: x64 | x86
  • Compiler: cl

Xbox

  • Generator: MSBuild
  • Architecture: x64
  • Compiler: cl

Linux

  • Generator: Ninja
  • Architecture: x64 | x86
  • Compiler: clang

MacOS

  • Generator: Ninja | Xcode
  • Architecture: x64 | x86
  • Compiler: clang

There’s more for Android and iOS, but it looks similar to the above. the point is, there’s a lot of configurations. You can’t really do a lot of complicated logic in a toolchain file because so many variables are not available, so the way I’ve worked around this is to have a different toolchain file for every combination. This means I need

WindowsDesktopNinjaX86Cl.cmake
WindowsDesktopNinjaX86ClangCl.cmake
WindowsDesktopNinjaX64Cl.cmake
WindowsDesktopNinjaX64ClangCl.cmake
WindowsDesktopMSBuildX86Cl.cmake
WindowsDesktopMSBuildX64Cl.cmake

So I have to multiply out the entire matrix of configurations to generate all toolchain files, which I’m not sure is better than just putting logic directly into CML after the project() command, where at least I can use loops and control flow statements to perform the logic. Am I missing a more elegant way to do this?

I know this is a lot, would appreciate some guidance though :slight_smile:

Cheers

2

bump

Same question here. @Zachary_Turner , did you get an answer offline, or find other resources to answer your questions?

3

:100::exclamation:

Thank you for posting this, and I’m really sad nobody answered. The question that brought me here is even more basic: what’s the practical difference between using a --toolchain file and passing settings on the command line (and, for that matter, putting them in a -C cache file)? I realize that they have different intended purposes, but will something stop working, or work differently, if I use one vs. the other?