CMake fails with: "The C compiler '../cl.exe' is not able to compile a simple test program"

I tried using cmake & ninja to build a simple project of mine and while it wasn’t an issue before when I was doing it with visual studio, I am now unable to do it with either the IDE or the command line because it fails with error message of the title.
The full output of the command I wrote is:

-- The C compiler identification is MSVC 19.37.32824.0
-- The CXX compiler identification is MSVC 19.37.32824.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe - broken
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:67 (message):
  The C compiler

    "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-a1hm2l

    Run Build Command(s):C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe -v cmTC_ca7b4 && [1/2] C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx86\x86\cl.exe  /nologo   /DWIN32 /D_WINDOWS /W3  /MDd /Ob0 /Od /RTC1 -Zi /showIncludes /FoCMakeFiles\cmTC_ca7b4.dir\testCCompiler.c.obj /FdCMakeFiles\cmTC_ca7b4.dir\ /FS -c "C:\Users\aggelos\Documents\Visual Studio 2022\Projects\mycmake\out\CMakeFiles\CMakeScratch\TryCompile-a1hm2l\testCCompiler.c"
    [2/2] cmd.exe /C "cd . && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_ca7b4.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx86\x86\link.exe /nologo CMakeFiles\cmTC_ca7b4.dir\testCCompiler.c.obj  /out:cmTC_ca7b4.exe /implib:cmTC_ca7b4.lib /pdb:cmTC_ca7b4.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
    FAILED: cmTC_ca7b4.exe
    cmd.exe /C "cd . && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_ca7b4.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx86\x86\link.exe /nologo CMakeFiles\cmTC_ca7b4.dir\testCCompiler.c.obj  /out:cmTC_ca7b4.exe /implib:cmTC_ca7b4.lib /pdb:cmTC_ca7b4.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
    ninja: build stopped: subcommand failed.

and the command was just cmake -G Ninja -DCMAKE_C_COMPILER="cl.exe" -DCMAKE_CXX_COMPILER="cl.exe" ...
Note that I did run vcvarsall.bat before running the cmake command.

Is there any more information in CMakeFiles/CMakeError.log? If not, does anything else show up when you pass the --debug-trycompile flag?

There is no CMakeError.log file but there is a CMakeConfigureLog.yml file.
And yes passing --debug-trycompile does print some extra info and it’s the following:

CMake Debug Log at C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeDetermineCompilerABI.cmake:57 (try_compile):
  Executing try_compile (CMAKE_C_ABI_COMPILED) in:

    C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-k7qi6x
Call Stack (most recent call first):
  C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:26 (CMAKE_DETERMINE_COMPILER_ABI)
  CMakeLists.txt:12 (project)


-- Detecting C compiler ABI info - failed
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe
CMake Debug Log at C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:56 (try_compile):
  Executing try_compile (CMAKE_C_COMPILER_WORKS) in:

    C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-gpktpz
Call Stack (most recent call first):
  CMakeLists.txt:12 (project)


-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe - broken

but I can’t really tell much from it

That .yml file (with the --debug-trycompile flag) may have better error info; alas the raw output here isn’t that much more useful.

At the moment it seems like I cannot upload the file but the important part is basically:

-
    kind: "try_compile-v1"
    backtrace:
      - "C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeDetermineCompilerABI.cmake:57 (try_compile)"
      - "C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:26 (CMAKE_DETERMINE_COMPILER_ABI)"
      - "CMakeLists.txt:12 (project)"
    checks:
      - "Detecting C compiler ABI info"
    directories:
      source: "C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-vnbcsx"
      binary: "C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-vnbcsx"
    cmakeVariables:
      CMAKE_C_FLAGS: "/DWIN32 /D_WINDOWS /W3"
      CMAKE_C_FLAGS_DEBUG: "/MDd /Ob0 /Od /RTC1"
      CMAKE_EXE_LINKER_FLAGS: "/machine:X86"
      CMAKE_MSVC_DEBUG_INFORMATION_FORMAT: "$<IF:$<AND:$<C_COMPILER_ID:MSVC>,$<CXX_COMPILER_ID:MSVC>>,$<$<CONFIG:Debug,RelWithDebInfo>:EditAndContinue>,$<$<CONFIG:Debug,RelWithDebInfo>:ProgramDatabase>>"
    buildResult:
      variable: "CMAKE_C_ABI_COMPILED"
      cached: true
      stdout: |
        Change Dir: C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-vnbcsx
        
        Run Build Command(s):C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe -v cmTC_b3aa0 && [1/2] C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\cl.exe  /nologo   /DWIN32 /D_WINDOWS /W3  /MDd /Ob0 /Od /RTC1 -Zi /showIncludes /FoCMakeFiles\\cmTC_b3aa0.dir\\CMakeCCompilerABI.c.obj /FdCMakeFiles\\cmTC_b3aa0.dir\\ /FS -c "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\share\\cmake-3.26\\Modules\\CMakeCCompilerABI.c"
        [2/2] cmd.exe /C "cd . && "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin\\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\\cmTC_b3aa0.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\rc.exe --mt=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\mt.exe --manifests  -- C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\link.exe /nologo CMakeFiles\\cmTC_b3aa0.dir\\CMakeCCompilerABI.c.obj  /out:cmTC_b3aa0.exe /implib:cmTC_b3aa0.lib /pdb:cmTC_b3aa0.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
        FAILED: cmTC_b3aa0.exe 
        cmd.exe /C "cd . && "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin\\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\\cmTC_b3aa0.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\rc.exe --mt=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\mt.exe --manifests  -- C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\link.exe /nologo CMakeFiles\\cmTC_b3aa0.dir\\CMakeCCompilerABI.c.obj  /out:cmTC_b3aa0.exe /implib:cmTC_b3aa0.lib /pdb:cmTC_b3aa0.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
        ninja: build stopped: subcommand failed.
        
      exitCode: 1
  -
    kind: "try_compile-v1"
    backtrace:
      - "C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:56 (try_compile)"
      - "CMakeLists.txt:12 (project)"
    checks:
      - "Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx86/x86/cl.exe"
    directories:
      source: "C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-tlo4ii"
      binary: "C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-tlo4ii"
    cmakeVariables:
      CMAKE_C_FLAGS: "/DWIN32 /D_WINDOWS /W3"
      CMAKE_C_FLAGS_DEBUG: "/MDd /Ob0 /Od /RTC1"
      CMAKE_EXE_LINKER_FLAGS: "/machine:X86"
      CMAKE_MSVC_DEBUG_INFORMATION_FORMAT: "$<IF:$<AND:$<C_COMPILER_ID:MSVC>,$<CXX_COMPILER_ID:MSVC>>,$<$<CONFIG:Debug,RelWithDebInfo>:EditAndContinue>,$<$<CONFIG:Debug,RelWithDebInfo>:ProgramDatabase>>"
    buildResult:
      variable: "CMAKE_C_COMPILER_WORKS"
      cached: true
      stdout: |
        Change Dir: C:/Users/aggelos/Documents/Visual Studio 2022/Projects/mycmake/out/CMakeFiles/CMakeScratch/TryCompile-tlo4ii
        
        Run Build Command(s):C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe -v cmTC_41890 && C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\cl.exe  /nologo   /DWIN32 /D_WINDOWS /W3  /MDd /Ob0 /Od /RTC1 -Zi /showIncludes /FoCMakeFiles\\cmTC_41890.dir\\testCCompiler.c.obj /FdCMakeFiles\\cmTC_41890.dir\\ /FS -c "C:\\Users\\aggelos\\Documents\\Visual Studio 2022\\Projects\\mycmake\\out\\CMakeFiles\\CMakeScratch\\TryCompile-tlo4ii\\testCCompiler.c"
        cmd.exe /C "cd . && "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin\\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\\cmTC_41890.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\rc.exe --mt=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\mt.exe --manifests  -- C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\link.exe /nologo CMakeFiles\\cmTC_41890.dir\\testCCompiler.c.obj  /out:cmTC_41890.exe /implib:cmTC_41890.lib /pdb:cmTC_41890.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
        FAILED: cmTC_41890.exe 
        cmd.exe /C "cd . && "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin\\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\\cmTC_41890.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\rc.exe --mt=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\mt.exe --manifests  -- C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\link.exe /nologo CMakeFiles\\cmTC_41890.dir\\testCCompiler.c.obj  /out:cmTC_41890.exe /implib:cmTC_41890.lib /pdb:cmTC_41890.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
        ninja: build stopped: subcommand failed.
        
      exitCode: 1

I should note that the cmake version I am using is the one that ships with VS and by running cmake --version it prints out cmake version 3.26.4-msvc4
Not sure if the above helps

It looks like something about the linking is failing. I do see /machine:X86 in there…are you compiling for 32-bit? Does it work if you don’t pass CMAKE_<LANG>_COMPILER settings manually and just let CMake discover the compiler from the vcvarsall procedure?

Yes I am targeting windows 32 bit at the moment so I run vcvarsall x86.
And yes I just tried invoking cmake without the hardcoded compilers and I still get the exact same response.
I also tried hardcoding clang-cl instead of cl to make it use clang and no difference.
I don’t know if it’s possible to make it display some useful info from the compiler because with the current info it seems like I can’t do much.

Can you link a simple executable yourself by hand at the command line? If not, it seems like some component is missing from the Visual Studio Installer (SDK? Build tools?).

Ok so I did run both cl.exe and link.exe (msvc compiler & linker) to build a simple project consisting of a single source file and it worked.
However running the linker with cmake using the command logged for my simple project:
C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin\\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\\cmTC_41890.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\rc.exe --mt=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x86\\mt.exe --manifests -- C:\\PROGRA~1\\MICROS~4\\2022\\COMMUN~1\\VC\\Tools\\MSVC\\1437~1.328\\bin\\Hostx86\\x86\\link.exe /nologo CMakeFiles\\cmTC_41890.dir\\testCCompiler.c.obj /out:cmTC_41890.exe /implib:cmTC_41890.lib /pdb:cmTC_41890.pdb /version:0.0 /machine:X86 /debug /INCREMENTAL /subsystem:console kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd .
It outputs nothing. No exe file or error.
I am not sure what this means

I can provide any information needed on how I built my simple project but it shouldn’t make much difference from how cmake tried to build it. The issue is most likely something with that cmake command that outputs nothing and I need to find out what tools are broken in my current installation

That’s probably what CMake is reacting to. Figuring out why there’s no executable would be the main thing. Can your link.exe make an executable if you remove CMake from the process?

Can your link.exe make an executable if you remove CMake from the process?

Yeah running link.exe directly works as expected.
The cmake command that is supposed to invoke link.exe (at least from what I can tell) takes -E vs_link_exe as argument which I haven’t yet figured out what it is for.
I will try and see if debugging helps

It’s worth noting that it looks like Microsoft is back to shipping its own customised version of CMake again :weary:. That is not an official release (the -msvc4 suffix tells us that), so it is also possible, though I suspect unlikely here, that the problem is in whatever they changed.

Maybe try with an official CMake release, just to eliminate that possibility from the picture.

Unfortunately that was one of the first things I tried.
I found out where the cmake binaries were stored in VS and I just replaced them with the latest official release of cmake.
And I got exactly the same output.
So I doubt cmake has anything to do with any of this

I experienced this being caused by the Windows Defender deleting the created .exe due to a false positive detection. You could try to disable your AV Software temporarily and run CMake again.

I only have Windows Defender and I have excluded the directories from which the .exe is generated so it shouldn’t be a problem

Alright so I passed --debug-trycompile to cmake and copy pasted the commands cmake run to the generated directory and it seems like ninja fails without any logs and for no reason?
I first tried running the command:
C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe -v cmTC_f8f73
the build files were all there and the output was:

[1/1] cmd.exe /C "cd . && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_f8f73.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx86\x86\link.exe /nologo CMakeFiles\cmTC_f8f73.dir\testCCompiler.c.obj  /out:cmTC_f8f73.exe /implib:cmTC_f8f73.lib /pdb:cmTC_f8f73.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
FAILED: cmTC_f8f73.exe
cmd.exe /C "cd . && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_f8f73.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx86\x86\link.exe /nologo CMakeFiles\cmTC_f8f73.dir\testCCompiler.c.obj  /out:cmTC_f8f73.exe /implib:cmTC_f8f73.lib /pdb:cmTC_f8f73.pdb /version:0.0 /machine:X86  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
ninja: build stopped: subcommand failed.

which clearly states that one of the subcommands failed and therefore I proceeded to invoke them seperately.
And to my surprise it worked as expected. The .exe file is generated with no issues.
I will see if downgrading ninja helps but this doesn’t make any sense.
And the fact that ninja outputs nothing useful is weird

Does it work if you call it with the && chaining? I wonder if there’s a race. Alternatively, watching for errors in procmon or some similar tool could shed light into what’s going on.

Nah. Thankfully I was right.
Ninja was indeed the issue.
I noticed that the ninja build files generated were targeting ninja 1.5 when I was building with ninja 1.11.
So I downgraded to 1.6 and to my surprise, it worked!
there is still only one issue left which is the fact that VS runs command with cmd /c "commands..." and for some reason that always returns 1 regardless of whether the commands succeeded or not.
But other than that it works perfectly fine and I was able to build the project from the command line using vcvarsall.

There probably has to be some breaking change on newer ninja releases, I have no idea

Interesting. Does it work if CMAKE_MAKE_PROGRAM agrees with the actual ninja you use to build?

It turns out I have made a mistake.
Newer ninja versions work fine.
What was really the problem was a registry value I added that runs a script when cmd.exe is run and the fact that this script is broken caused cmd.exe to always return error code 1.
After deleting the value everything worked as expected both from the command line and Visual Studio.

I apologize for the confusion