I am currently exploring the presets feature. I am trying to make a windows, ninja, clang-cl based build work.
I managed to do it but it only works if I call Visual Studios vcvarsall.bat file first to get the correct environment.
I never liked having to make the extra call to that file though, so I am wondering if there is a way to embedd that call into the CMakePresets.json file or maybe set the environment by hand in that file. So here are my questions:
Is it possible to specify running a command in the CMakePresets.json file?
Is there an working example of an CMakePresets.json that sets the same environment as the vcvarsall.bat file?
Side question… Why is it not necessary to call vsvarsall.bat when using the visual studio generator?
No, it has always been the job of either the generator (Visual Studio and Xcode do it) or the caller to set up the environment properly to use the compiler. CMake is not a cmd.exe program and cannot read vcvarsall.bat meaningfully without implementing the .bat language itself.
CMake isn’t responsible for setting up your compiler environment. It tries to help as much as it can but it’s an unreasonable burden to keep with the decision of the Visual Studio team. If they did decide to parse those bat files, and then visual studio 2021 comes out and breaks it, then it’s wasted effort.
Instead if you want an easy Ninja + VS + Clang environment then you basically have to do what the VS team wants you to do. CMakeSettings.json is the VS team’s answer to people who want the workflow you just expressed.
The x64 versus arm64 stuff seems to be handled pretty well by the CMakeSettings.json configuration “inheritEnvironments” value, but there are no defined values for inheritEnvironments to set up the build for UWP.
I ended up writing a big BAT file and invoking it from add_custom_target in CMakeLists.txt
That BAT then does a CALL VCVARSALL.BAT to set up the environment – including the UWP argument, and then calls ninja to rebuild with that environment (with the proper library paths etc.). To protect itself from recursion, my BAT immediately looks for a _BUILD_RECURSION_GUARD.TXT file. If it exists, the BAT exits immediately. If it doesn’t exit, the BAT creates it, then finally deletes it at the end.
A total ugly kluge, but it seems to work. I have not found a better way the set up the VCVARSALL environment for UWP in a Visual Studio CMake project. I’ve done a whole lot of BAT scripting to work around various problems.
It just looks like CMakeSettings.json and CMakePresets.json have a lot of overlap.
I am not sure if it makes sense to use both.
In the end all I am looking for is a platform and config independent build interface. Having to call
vcvarsall.bat before running cmake --build forces me to have another buildscript that wraps the cmake call and runs vcvarsall.bat whenever there is a windows-ninja configuration. This is a bit ugly and I am looking for a way to move that code into the cmake generate step so the user only has to select a configuration file and everything works.
CMakeSettings.json can not accomplish that because it is that wrapper around the call to cmake generate but it is only used by visual studio while I would like to keep my build-setup general enough to support all generators.
I am not completely understanding your setup here.
But maybe you can run the cmake generate step recursively with an execute_process(). But it sounds really ugly. I would rather have a clean built-in solution that is provided by cmake.
But would it somehow be possible for cmake to run the vcvarsall.bat itself with an execute_process() and get the changes in the environment and use it for it’s own environment?
The outline of my ugly embarrassing hack for compiling and linking UWP – because the Visual Studio CMakeSettings.json mechanism doesn’t include any provision for passing the “uwp” argument to vcvarsall.bat:
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
IF EXIST !BUILD_RECURSION_GUARD! (
EXIT /B 0
)
ECHO Recursion Guard >!BUILD_RECURSION_GUARD!
REM : Iterate through { x64 | arm64 } x { Debug | Release } configurations
REM : Re-initialize INCLUDE, LIB, LIBPATH, and PATH -- because vcvarsall.bat
REM : doesn't replace then, it appends them. Then call vcvarsall.bat to set up the
REM : environment, then ninja to build.
REM : Note the protection above against ninja recursing into this bat file.
CALL "!VCVARSALL!" !BUILD_ARCH! uwp !BUILD_WINSDKVER!
ninja -v clean
ninja -v all
REM : Then do other stuff to build a UWP MSIX package file.
REM : Then clean up.
IF EXIST !BUILD_RECURSION_GUARD! (
DEL !BUILD_RECURSION_GUARD!
)
Even if CMake could, that only solves it for the CMake process itself. nmake and ninja certainly won’t do this, so you still need to load the environment to do the actual build (thoughcmake --build could do it I suppose, but we’d first need “load the right compiler env” to be a thing in the first place).
I will also say that this is not a Windows-only problem. For example, on HPC machines, compiler environments must be manually loaded in order to function CMake does not support anything other than “bring your own compiler env” anywhere. It’s just that the compiler environment for “standard” compilers on *nix machines are “trivial” (either cc “just working” or CC/CXX environment variables) and work out of the box (Xcode needs a little hint with DEVELOPER_DIR, but installing the CommandLineTools stuff also works).
Ah ok, but how does it work when using the Visual Studio Generator? CMake lets us choose the bitness of the build so it seems to have some hand into choosing the right compiler or environment.
Ok I finally find a way to detect and use msvc compiler automatically in cmake.
The result looks like a toolchain file (as far as a toolchain files setup some compiler variables and utility path…), can be used as a toolchain file (occuring multiple calls)
or can be simply included in your cmake scripts before any project statement, maybe under a if (NOT CMAKE_GENERATOR MATCHES "Visual Studio") condition to avoid another useless call.
capture the result of “set” command by running it with the vcvars launcher (“set” output all env vars)
reinject all the values in current process executing cmake, by using set(ENV{key} value)
feed cmake with include dirs and libs :
set(CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES $ENV{INCLUDE})
link_directories($ENV{LIB})
et voila.
Used succesfully with Ninja/msvc. Enhance immediatly presets interest in this case !
Implementation details :
in findvcvars, the calls to cmake_host_system_information (used to iteratively search for msvc version from newer to older) can output an error, depending on cmake version.
So I patched it by calling the command through an execute_process to eat the error and continue searching. Otherwise some IDE (like Visual Studio) found that an error occured and you will never be able to build anything.
2/3. Of course parsing the output of “set” and driving it correctly to $ENV is tricky…
As a user, I’m not able to join files so I include here the three needed cmake files.
msvcToolchain.cmake
# allow findXXX resolution in CMake/
list(INSERT CMAKE_MODULE_PATH 0 ${CMAKE_CURRENT_LIST_DIR})
find_package(Vcvars)
if (NOT ${Vcvars_FOUND})
message(FATAL_ERROR "[msvcToolchain.cmake] Looks like no Visual Studio installed")
endif()
# got vcvars launcher !
set(cmd_wrapper ${Vcvars_LAUNCHER})
# DEBUG : dump env after vcvars*.bat call
#execute_process(COMMAND ${cmd_wrapper} set OUTPUT_FILE ${CMAKE_CURRENT_LIST_DIR}/debug/out.env.before.txt OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(COMMAND ${cmd_wrapper} set OUTPUT_VARIABLE env_dump OUTPUT_STRIP_TRAILING_WHITESPACE)
#1. escaping troublesome chars
string(REPLACE ";" "__semicolon__" env_dump "${env_dump}")
string(REPLACE "\\" "__backslash__" env_dump "${env_dump}")
string(REPLACE "\"" "__doublequote__" env_dump "${env_dump}")
#2. multi-line => one line
string(REGEX REPLACE "[\r\n]+" ";" env_dump "${env_dump}")
#3. keep only lines looking like xx=yy
list(FILTER env_dump INCLUDE REGEX ".+\=.+")
#4. setting captured env var right here
foreach(key_value ${env_dump})
string(REPLACE "=" ";" key_value_as_list ${key_value})
list(GET key_value_as_list 0 key)
list(GET key_value_as_list 1 value)
string(REPLACE "__semicolon__" "\;" key "${key}")
string(REPLACE "__backslash__" "\\" key "${key}")
string(REPLACE "__doublequote__" "\"" key "${key}")
string(REPLACE "__semicolon__" ";" value "${value}")
string(REPLACE "__backslash__" "\\" value "${value}")
string(REPLACE "__doublequote__" "\"" value "${value}")
set(ENV{${key}} "${value}")
endforeach()
#5. adjust cmake vars to find msvc headers & resolve libs path
set(CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES $ENV{INCLUDE})
link_directories($ENV{LIB})
message(STATUS "[msvcToolchain.cmake] env dumping done")
# DEBUG : dump en var after all this
# execute_process(COMMAND cmd /C set OUTPUT_FILE ${CMAKE_CURRENT_LIST_DIR}/debug/out.env.after.txt OUTPUT_STRIP_TRAILING_WHITESPACE)
findVCVars.cmake
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.
#[=======================================================================[.rst:
FindVcvars
----------
Finds a "vcvars" batch script.
The module can be used when configuring a project or when running
in cmake -P script mode.
These variables can be used to choose which "vcvars" batch script is looked up.
.. variable:: Vcvars_MSVC_ARCH
Possible values are `32` or `64`
If not explicitly set in the calling scope, the variable is initialized
based on the value of :variable:`CMAKE_SIZEOF_VOID_P` in configuration mode, and
to 64 in script mode.
.. variable:: Vcvars_MSVC_VERSION
Possible values corresponds to :variable:`MSVC_VERSION`.
If not explicitly set in the calling scope, :variable:`Vcvars_MSVC_VERSION` is
initialized using :variable:`MSVC_VERSION` variable if it is defined, otherwise
the variable :variable:`Vcvars_MSVC_VERSION` is initialized based on the most
recent version of Visual Studio installed on the system.
This will define the following variables:
.. variable:: Vcvars_BATCH_FILE
Path to ``vcvars32.bat``, ``vcvarsamd64.bat`` or ``vcvars64.bat``.
.. variable:: Vcvars_LAUNCHER
Path to a generated wrapper script allowing to execute program after
setting environment defined by `Vcvars_BATCH_FILE`.
It can be used within :module:`ExternalProject` steps
specifying command like this::
set(cmd_wrapper)
if(MSVC)
find_package(Vcvars REQUIRED)
set(cmd_wrapper ${Vcvars_LAUNCHER})
endif()
ExternalProject_Add(AwesomeProject
[...]
BUILD_COMMAND ${cmd_wrapper} <command> arg1 arg2 [...]
[...]
)
This module also defines the following functions
.. command:: Vcvars_GetVisualStudioPaths
The ``Vcvars_GetVisualStudioPaths()`` function returns a list of all
possible Visual Studio registry paths associated with a given ``<msvc_version>``
and ``<msvc_arch>``::
Vcvars_GetVisualStudioPaths(<msvc_version> <msvc_arch> <output_var>)
The options are:
``<msvc_version>``
Specify the Visual Studio compiler version. See :variable:`MSVC_VERSION`
for possible values.
``<msvc_arch>``
Specify the Visual Studio architecture. Possible values are `32` or `64`.
``<output_var>``
The name of the variable to be set with the list of registry paths.
.. command:: Vcvars_ConvertMsvcVersionToVsVersion
The ``Vcvars_ConvertMsvcVersionToVsVersion()`` function converts a
:variable:`MSVC_VERSION` of the form ``NNNN`` to a Visual Studio version
of the form ``XX.Y`::
Vcvars_ConvertMsvcVersionToVsVersion(<msvc_version> <output_var>)
The options are:
``<msvc_version>``
Specify the Visual Studio compiler version. See :variable:`MSVC_VERSION`
for possible values.
``<output_var>``
The name of the variable to be set with the Visual Studio version.
#]=======================================================================]
cmake_minimum_required(VERSION 3.5)
# TODO Support lookup of "Microsoft Visual C++ Compiler for Python 2.7 (x86, amd64)"
# Global variables used only in this script (unset at the end)
set(_Vcvars_MSVC_ARCH_REGEX "^(32|64)$")
set(_Vcvars_MSVC_VERSION_REGEX "^[0-9][0-9][0-9][0-9]$")
set(_Vcvars_SUPPORTED_MSVC_VERSIONS 1930
1929 1928 1927 1926 1925 1924 1923 1922 1921 1920
1916 1915 1914 1913 1912 1911 1910 1900
1800 1700 1600 1500 1400)
function(_vcvars_message)
if(NOT Vcvars_FIND_QUIETLY)
message(${ARGN})
endif()
endfunction()
function(Vcvars_ConvertMsvcVersionToVsVersion msvc_version output_var)
if(NOT msvc_version MATCHES ${_Vcvars_MSVC_VERSION_REGEX})
message(FATAL_ERROR "msvc_version is expected to match `${_Vcvars_MSVC_VERSION_REGEX}`")
endif()
# See https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering
if((msvc_version GREATER_EQUAL 1930) AND (msvc_version LESS 1940)) # VS 2022
set(vs_version "17")
elseif((msvc_version GREATER_EQUAL 1920) AND (msvc_version LESS 1930)) # VS 2019
set(vs_version "16")
elseif((msvc_version GREATER_EQUAL 1910) AND (msvc_version LESS 1920)) # VS 2017
set(vs_version "15")
elseif(msvc_version EQUAL 1900) # VS 2015
set(vs_version "14.0")
elseif(msvc_version EQUAL 1800) # VS 2013
set(vs_version "12.0")
elseif(msvc_version EQUAL 1700) # VS 2012
set(vs_version "11.0")
elseif(msvc_version EQUAL 1600) # VS 2010
set(vs_version "10.0")
elseif(msvc_version EQUAL 1500) # VS 2008
set(vs_version "9.0")
elseif(msvc_version EQUAL 1400) # VS 2005
set(vs_version "8.0")
elseif(msvc_version EQUAL 1310) # VS 2003
set(vs_version "7.1")
elseif(msvc_version EQUAL 1300) # VS 2002
set(vs_version "7.0")
elseif(msvc_version EQUAL 1200) # VS 6.0
set(vs_version "6.0")
else()
message(FATAL_ERROR "failed to convert msvc_version [${msvc_version}]. It is not a known version number.")
endif()
set(${output_var} ${vs_version} PARENT_SCOPE)
endfunction()
function(Vcvars_GetVisualStudioPaths msvc_version msvc_arch output_var)
if(NOT msvc_version MATCHES ${_Vcvars_MSVC_VERSION_REGEX})
message(FATAL_ERROR "msvc_version is expected to match `${_Vcvars_MSVC_VERSION_REGEX}`")
endif()
if(NOT msvc_arch MATCHES ${_Vcvars_MSVC_ARCH_REGEX})
message(FATAL_ERROR "msvc_arch argument is expected to match '${_Vcvars_MSVC_ARCH_REGEX}'")
endif()
Vcvars_ConvertMsvcVersionToVsVersion(${msvc_version} vs_version)
set(_vs_installer_paths "")
set(_vs_registry_paths "")
if(vs_version VERSION_GREATER_EQUAL "15.0")
# Query the VS Installer tool for locations of VS 2017 and above.
string(REGEX REPLACE "^([0-9]+)\.[0-9]+$" "\\1" vs_installer_version ${vs_version})
#make the cmake_host_system_information call in another process to filter expected error on the search a visual studio compiler
# case: a version not supported by the command itself)
# cmake_host_system_information(RESULT _vs_dir QUERY VS_${vs_installer_version}_DIR)
# => cmake_host_system_information is deferred in an execute_process call to absorb errors
set(arg_key VS_${vs_installer_version}_DIR)
set(configured_script cmake_host_system_information.${vs_installer_version}.cmake)
configure_file(
"${CMAKE_CURRENT_LIST_DIR}/cmake_host_system_information.cmake.in"
"${CMAKE_CURRENT_LIST_DIR}/${configured_script}"
@ONLY
)
unset(_vs_dir)
execute_process(COMMAND ${CMAKE_COMMAND} -Darg_key=VS_${vs_installer_version}_DIR -P ${CMAKE_CURRENT_LIST_DIR}/${configured_script} OUTPUT_VARIABLE _vs_dir OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET)
if(_vs_dir)
list(APPEND _vs_installer_paths "${_vs_dir}/VC/Auxiliary/Build")
endif()
else()
# Registry keys for locations of VS 2015 and below
set(_hkeys
"HKEY_USERS"
"HKEY_CURRENT_USER"
"HKEY_LOCAL_MACHINE"
"HKEY_CLASSES_ROOT"
)
set(_suffixes
""
"_Config"
)
set(_arch_path "bin/amd64")
if(msvc_arch STREQUAL "32")
set(_arch_path "bin")
endif()
set(_vs_registry_paths)
foreach(_hkey IN LISTS _hkeys)
foreach(_suffix IN LISTS _suffixes)
set(_vc "VC")
if(_vs_version STREQUAL "6.0")
set(_vc "Microsoft Visual C++")
endif()
list(APPEND _vs_registry_paths
"[${_hkey}\\SOFTWARE\\Microsoft\\VisualStudio\\${vs_version}${_suffix}\\Setup\\${_vc};ProductDir]/${_arch_path}"
)
endforeach()
endforeach()
endif()
set(_vs_installer_paths ${_vs_installer_paths} ${_vs_registry_paths})
if(_vs_installer_paths STREQUAL "")
set(_vs_installer_paths "${output_var}-${msvc_version}-${msvc_arch}-NOTFOUND")
endif()
set(${output_var} ${_vs_installer_paths} PARENT_SCOPE)
endfunction()
# default
if(NOT DEFINED Vcvars_MSVC_ARCH)
if(NOT DEFINED CMAKE_SIZEOF_VOID_P)
set(Vcvars_MSVC_ARCH "64")
_vcvars_message(STATUS "Setting Vcvars_MSVC_ARCH to '${Vcvars_MSVC_ARCH}' as CMAKE_SIZEOF_VOID_P was none")
else()
if("${CMAKE_SIZEOF_VOID_P}" EQUAL 4)
set(Vcvars_MSVC_ARCH "32")
elseif("${CMAKE_SIZEOF_VOID_P}" EQUAL 8)
set(Vcvars_MSVC_ARCH "64")
else()
message(FATAL_ERROR "CMAKE_SIZEOF_VOID_P [${CMAKE_SIZEOF_VOID_P}] is expected to be either 4 or 8")
endif()
# Display message only once in config mode
if(NOT DEFINED Vcvars_BATCH_FILE)
_vcvars_message(STATUS "Setting Vcvars_MSVC_ARCH to '${Vcvars_MSVC_ARCH}' as CMAKE_SIZEOF_VOID_P was `${CMAKE_SIZEOF_VOID_P}`")
endif()
endif()
endif()
if(NOT DEFINED Vcvars_MSVC_VERSION)
if(DEFINED MSVC_VERSION)
set(Vcvars_MSVC_VERSION ${MSVC_VERSION})
# Display message only once in config mode
if(NOT DEFINED Vcvars_BATCH_FILE)
_vcvars_message(STATUS "Setting Vcvars_MSVC_VERSION to '${Vcvars_MSVC_VERSION}' as MSVC_VERSION was `${MSVC_VERSION}`")
endif()
endif()
endif()
if(NOT DEFINED Vcvars_BATCH_FILE)
set(Vcvars_BATCH_FILE "Vcvars_BATCH_FILE-NOTFOUND")
endif()
if(NOT DEFINED Vcvars_LAUNCHER)
set(Vcvars_LAUNCHER "Vcvars_LAUNCHER-NOTFOUND")
endif()
# check Vcvars_MSVC_ARCH is propertly set
if(NOT Vcvars_MSVC_ARCH MATCHES ${_Vcvars_MSVC_ARCH_REGEX})
message(FATAL_ERROR "Vcvars_MSVC_ARCH [${Vcvars_MSVC_ARCH}] is expected to match `${_Vcvars_MSVC_ARCH_REGEX}`")
endif()
# which vcvars script ?
if(Vcvars_MSVC_ARCH STREQUAL "64")
set(_Vcvars_SCRIPTS vcvarsamd64.bat vcvars64.bat)
else()
set(_Vcvars_SCRIPTS vcvars32.bat)
endif()
# set Vcvars_BATCH_FILE
if(NOT DEFINED Vcvars_MSVC_VERSION)
# auto-discover Vcvars_MSVC_VERSION value
_vcvars_message(STATUS "Setting Vcvars_MSVC_VERSION")
foreach(_candidate_msvc_version IN LISTS _Vcvars_SUPPORTED_MSVC_VERSIONS)
Vcvars_GetVisualStudioPaths(${_candidate_msvc_version} "${Vcvars_MSVC_ARCH}" _paths)
Vcvars_ConvertMsvcVersionToVsVersion(${_candidate_msvc_version} _candidate_vs_version)
set(_msg " Visual Studio ${_candidate_vs_version} (${_candidate_msvc_version})")
_vcvars_message(STATUS "${_msg}")
unset(Vcvars_BATCH_FILE CACHE)
find_program(Vcvars_BATCH_FILE NAMES ${_Vcvars_SCRIPTS}
DOC "Visual Studio ${_candidate_vs_version} ${_Vcvars_SCRIPTS}"
PATHS ${_paths}
)
if(Vcvars_BATCH_FILE)
_vcvars_message(STATUS "${_msg} - found")
set(Vcvars_MSVC_VERSION ${_candidate_msvc_version})
_vcvars_message(STATUS "Setting Vcvars_MSVC_VERSION to '${Vcvars_MSVC_VERSION}' as it was the newest Visual Studio installed providing vcvars scripts")
break()
else()
_vcvars_message(STATUS "${_msg} - not found")
endif()
endforeach()
unset(_candidate_msvc_version)
unset(_candidate_vs_version)
unset(_paths)
else()
# use provided Vcvars_MSVC_VERSION value
if(NOT Vcvars_MSVC_VERSION MATCHES ${_Vcvars_MSVC_VERSION_REGEX})
message(FATAL_ERROR "Vcvars_MSVC_VERSION [${Vcvars_MSVC_VERSION}] is expected to match `${_Vcvars_MSVC_VERSION_REGEX}`")
endif()
Vcvars_GetVisualStudioPaths(${Vcvars_MSVC_VERSION} "${Vcvars_MSVC_ARCH}" _paths)
Vcvars_ConvertMsvcVersionToVsVersion(${Vcvars_MSVC_VERSION} _vs_version)
find_program(Vcvars_BATCH_FILE NAMES ${_Vcvars_SCRIPTS}
DOC "Visual Studio ${_vs_version} ${_Vcvars_SCRIPTS}"
PATHS ${_paths}
)
unset(_paths)
unset(_vs_version)
endif()
# configure wrapper script
set(Vcvars_LAUNCHER)
if(Vcvars_BATCH_FILE)
set(_in "${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/Vcvars_wrapper.bat.in")
get_filename_component(_basename ${Vcvars_BATCH_FILE} NAME_WE)
set(_out "${CMAKE_BINARY_DIR}/${CMAKE_FILES_DIRECTORY}/${_basename}_wrapper.bat")
file(WRITE ${_in} "call \"@Vcvars_BATCH_FILE@\"
%*
")
configure_file(${_in} ${_out} @ONLY)
set(Vcvars_LAUNCHER ${_out})
unset(_in)
unset(_out)
endif()
# cleanup
unset(_Vcvars_SCRIPTS)
# outputs
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Vcvars
FOUND_VAR Vcvars_FOUND
REQUIRED_VARS
Vcvars_BATCH_FILE
Vcvars_LAUNCHER
Vcvars_MSVC_VERSION
Vcvars_MSVC_ARCH
FAIL_MESSAGE
"Failed to find vcvars scripts for Vcvars_MSVC_VERSION [${Vcvars_MSVC_VERSION}] and Vcvars_MSVC_ARCH [${Vcvars_MSVC_ARCH}]"
)
user, I’m not able to join files so I include here the three needed cmake files.