Best Practice for Ninja build Visual Studio?

Usage
,
1

Note: CMAKE_MINIMUM_REQUIRED(VERSION 3.21)

What is the recommended way to configure/invoke CMake such that it will use Ninja as the build tool but still locate the compiler/etc the way it would if you had specified the Visual Studio Code generator to use msbuild?

This isn’t just about forgetting to start a developer command prompt/pwsh, but it’s very common from desktop applications using the system/user paths, such as VS Code:

[cmake] CMake Error at CMakeLists.txt:14 (PROJECT):
[cmake]   No CMAKE_CXX_COMPILER could be found.

To reproduce:

  1. Open a prompt and confirm it does not have the vsdev environment variables loaded, and you cannot run cl.exe
  2. Ensure that ninja is in your path (e.g nuget/choco install it, or copy your vsdev ninja into c:\windows or something)
  3. Ensure cmake is in your path
  4. Create a trivial Hello World project
  5. cmake -G Ninja . build
1 Like
2

The Ninja (really, all non-IDE generators) expect the environment to have the compilers set up already. You’ll need to load the compiler environment before you can use such generators. Granted, this is “easy” on Unix-like platforms since the compiler is usually set up system-wide, but this is not the case for the Visual Studio toolchains (among other compilers in various places and systems which use module load to make toolchains available).

3

I have the right environment but get this:

C:\Users\klein_cl\Workspace\cpp\minijson_reader>cmake -G Ninja -B build-ninja --check-system-vars --debug-trycompile
Also check system files when warning about unused and uninitialized variables.
debug trycompile on
-- The CXX compiler identification is MSVC 19.28.29914.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe - broken
CMake Error at C:/Program Files/CMake/share/cmake-3.23/Modules/CMakeTestCXXCompiler.cmake:62 (message):
  The C++ compiler

    "C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/klein_cl/Workspace/cpp/minijson_reader/build-ninja/CMakeFiles/CMakeTmp

    Run Build Command(s):C:/RsPython/scripts/ninja.exe cmTC_c0149 && [1/2] Building CXX object CMakeFiles\cmTC_c0149.dir\testCXXCompiler.cxx.obj
    [2/2] Linking CXX executable cmTC_c0149.exe
    FAILED: cmTC_c0149.exe
    cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_c0149.dir --rc=rc --mt=CMAKE_MT-NOTFOUND --manifests  -- C:\PROGRA~2\MIB055~1\2019\PROFES~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe /nologo CMakeFiles\cmTC_c0149.dir\testCXXCompiler.cxx.obj  /out:cmTC_c0149.exe /implib:cmTC_c0149.lib /pdb:cmTC_c0149.pdb /version:0.0 /machine:x64  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
    RC Pass 1: command "rc /fo CMakeFiles\cmTC_c0149.dir/manifest.res CMakeFiles\cmTC_c0149.dir/manifest.rc" failed (exit code 0) with the following output:
    Das System kann die angegebene Datei nicht finden
    ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:15 (project)


-- Configuring incomplete, errors occurred!
See also "C:/Users/klein_cl/Workspace/cpp/minijson_reader/build-ninja/CMakeFiles/CMakeOutput.log".
See also "C:/Users/klein_cl/Workspace/cpp/minijson_reader/build-ninja/CMakeFiles/CMakeError.log".

C:\Users\klein_cl\Workspace\cpp\minijson_reader>
4

I need to execute vcvars, than it works!

call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvars64.bat"
1 Like
5

You should also have a maybe more convenient direct command prompt shortcut (search for a native tools shortcut in the start menu) that calls vcvars for you, in my case it calls:

C:\Windows\System32\cmd.exe /k "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Auxiliary\Build\vcvars64.bat"

6

As a die-hard Unix/Linux developer, please give consideration to this feedback:

Unless you absolutely need to support cmd, I strongly encourage you to consider migrating to Powershell for interacting with Windows. I only intended to initially use it as a “Rosetta Stone” for bringing our Windows CI scripts up to par with Linux and Mac, but instead I discovered that by changing our jenkins code from

shell {
...
}

to

pwsh {
...
}

[almost] everything just worked.

(3) Powershell: Blitz Talk - YouTube

But in this context, you can enable the developer shell at any point by simply doing:

Import-Module Microsoft.VisualStudio.DevShell.dll
Enable-VSDevShell

I put the following in my $profile (global variable in ps) and then just use ‘vsdev’: Powershell macro to enable visual studio developer studio AND save your path (github.com)

7

Nifty. How do you select what version of VS and toolchain to load?

1 Like
9

For that you have to steer vswhere, and seems I’d taken that out of the script because I realized my implementation was a bit too noddy.

  # Get the most recent install path (-last 1) from its list of installed locations.
  $VisualStudioInstallPath = & $VsWhere -prerelease -latest -property installationPath

A powershell hint: PS stuck with the posix spec for cmd arguments to enable it to do really powerful autocompletion, but that means that arguments are always single - and help is ‘-?’

vswhere -?
Visual Studio Locator version 3.0.3+45247720e1 [query version 3.4.1128.26111]
Copyright (C) Microsoft Corporation. All rights reserved.

Usage:  [options]

Options:
...

This is my naive original version of that:

function vsdev () {
  Param([string] $Version = "latest")
  $VsWhere = Join-Path (Join-Path (Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio') Installer) vswhere.exe
  if ( Test-Path $VsWhere ) {
    if ($Version -eq "latest") {
      $VisualStudioInstallPath = & $VsWhere -prerelease -latest -property installationPath | select -last 1
    } else {
      $VisualStudioInstallPath = & $VsWhere -prerelease -version "[${Version}.0,$($Version+1).0" -property installationPath | select -last 1
    }
    Import-Module -ea stop (Join-Path (Join-Path (Join-Path $VisualStudioInstallPath Common7) Tools) Microsoft.VisualStudio.DevShell.dll)
    Enter-VsDevShell -VsInstallPath $VisualStudioInstallPath -SkipAutomaticLocation -DevCmdArguments "-arch=amd64 -host_arch=amd64"
  } else {
    write-error "Could not find ${vswhere}"
  }
}

But vswhere supports “-format” options that you could use to do the selection more naturally.

PS> vswhere -format json -prerelease | convertfrom-json
PS> $versions.length
1
PS> $versions[0]

instanceId          : d3a837e5
installDate         : 11/11/2022 4:40:49 AM
installationName    : VisualStudio/17.4.4+33213.308
installationPath    : C:\Program Files\Microsoft Visual Studio\2022\Community
installationVersion : 17.4.33213.308
...
PS> $versions[0].installationPath
C:\Program Files\Microsoft Visual Studio\2022\Community
PS> $versions | where installationVersion -eq 17

and then you can use more natural pattern matching:

PS> $want = '17.4.*' ; $versions | where installationVersion -match $want

instanceId          : d3a837e5
installDate         : 11/11/2022 4:40:49 AM
installationName    : VisualStudio/17.4.4+33213.308
installationPath    : C:\Program Files\Microsoft Visual Studio\2022\Community
installationVersion : 17.4.33213.308

=== annecdotes ===

It seems a lot of folks who have .net experience are more familiar with working with xml,

PS> [xml]$versions = & $vswhere -prerelease -format xml
# prefix '&' denotes a command line vs an operation on a variable
PS> $versions
xml           instances
---           ---------
version="1.0" instances
PS> $versions.instances.gettype()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    XmlElement                               System.Xml.XmlLinkedNode
PS> $versions | get-member

   TypeName: System.Xml.XmlDocument

Name                        MemberType            Definition
----                        ----------            ----------
ToString                    CodeMethod            static string XmlNode(psobject inst…
AppendChild                 Method                System.Xml.XmlNode AppendChild(Syst…
Clone                       Method                System.Xml.XmlNode Clone(), System.…
CloneNode                   Method                System.Xml.XmlNode CloneNode(bool d…
...
10

Perhaps you may checkout GitHub - MarkSchofield/WindowsToolchain: A repository containing a CMake toolchain for using MSVC

I supports building your project with ninja and MS visual studio compiler or with MS clang compiler.

2 Likes
11

Interesting! For your Q/A list: Does it support cl/clang-cl/clang?

Aside: I saw the ‘install vswhere if its not here’, which makes me question my understanding. Does that mean that there are cases where there is some useful part of the vs build tooling that can be installed without vswhere? I thought I had understood that “not exists $vshwere” was a legitimate datapoint.

12

If you allow internet access while build on your CI, it downloads evening it is missing, but requested by your project.

With this toolchain and ninja, it is the fasted way to build your program on windows for windows.

If your project needs to be portable, yous GitHub - aminya/project_options: A general-purpose CMake library that provides functions that improve the CMake experience following the best practices..

13

Thanks for the replies, Claus - I’m asking from curiosity, I hope that doesn’t get lost in translation thru English!

In order for the tool chain to work, they would need to first install cmake. I would not have thought to require a user to install only cmake instead installing msbuild tools, which includes cmake. Installing the build tools after cmake has started running seems like it would cause a lot of additional complexity, no? For instance, if cmake has already consumed a ‘project()’ declaration, then it may behave very differently on a second run?

I know this is the internet so we are only supposed to ask questions to be rude, but I ask these as questions I only had assumptions to and your project made me aware that I had assumed!

– Oliver

14

I saw the ‘install vswhere if its not here’, which makes me question my understanding. Does that mean that there are cases where there is some useful part of the vs build tooling that can be installed without vswhere? I thought I had understood that “not exists $vshwere” was a legitimate datapoint.

I don’t know if it’s possible to install VS without VSWhere, but the “Tools for detecting and managing Visual Studio instances” documentation doesn’t explicitly state that you can rely on C:\Program Files (x86)\Microsoft Visual Studio\Installer location. The documentation does reference the vswhere releases, and that’s where the WindowsToolchain pulls it from as a fallback to C:\Program Files (x86)\Microsoft Visual Studio\Installer.

In order for the tool chain to work, they would need to first install cmake.

That’s right, the toolchain is processed by CMake; CMake would need to be installed.

Installing the build tools after cmake has started running seems like it would cause a lot of additional complexity, no?

The “WindowsToolchain” toolchain will download Ninja if it can’t find it, mainly because Ninja doesn’t have a well-defined install location. The “WindowsToolchain” toolchain won’t download MSVC/Clang, it expects to be able to find it through vswhere.exe.

For instance, if cmake has already consumed a ‘project()’ declaration, then it may behave very differently on a second run?

A Toolchain is supposed to be processed before a project() declaration, isn’t it? In which case, I think it’s OK - ish? - to pull the Generator.

Does it support cl/clang-cl/clang?

It does.

2 Likes
15

Thank you, Mark. Very interesting.