So I’ve been trying to generate the cmake project for Mitsuba 2 on Windows 10 (build: 19041.572) using the Visual Studio 16 2019 generator and x64 toolchain.
This is a project that requires CUDA and thus for the architecture specification, I set the flag CMAKE_CUDA_ARCHITECTURES
flag to 75
which should be supported with CUDA SDK version 11.1.74 and my GPU (RTX 2070) that goes up to compute capability 7.5. Unfortunately, I’ve found that setting the CMAKE_CUDA_ARCHITECTURES
flag to 75
leads to a fatal error whereupon CMakeDetermineCUDACompiler.cmake
fails to detect this as a valid architecture. This gives the following error:
CMake Error at C:/Program Files/CMake/share/cmake-3.18/Modules/CMakeDetermineCUDACompiler.cmake:551 (message):
The CMAKE_CUDA_ARCHITECTURES:
75
do not all work with this compiler. Try:
52
instead.
Call Stack (most recent call first):
ext/enoki/CMakeLists.txt:28 (enable_language)
Normally, I would consider this an error on my part as maybe I’ve misconfigured something, but when I set the architecture to 50
I encounter the exact same error which makes no sense considering that the CUDA SDK still supports compute capability 5.0.
Thus I dug further into where this error occurs and found that in CMakeDetermineCompilerId.cmake
the Visual Studio + CUDA path (line 365 to 421) does not correctly set the architecture flags for the temporary project, build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj
, generated from ${CMAKE_ROOT}/Modules/CompilerId/VS-${v}.${ext}.in
. Essentially, in the call to
function(CMAKE_DETERMINE_COMPILER_ID_BUILD lang testflags userflags src)
the testflags
and userflags
are not correctly passed to the compiler when the ${CMAKE_GENERATOR}
is any version of Visual Studio.
You can also see this by looking at the CMakeOutput.log
generated during the project configuration:
Compiling the CUDA compiler identification source file "CMakeCUDACompilerId.cu" succeeded.
Compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/bin/nvcc.exe
Build flags: ;-gencode=arch=compute_75,code=compute_75
Id flags: --cuda-gpu-arch=sm_75 -v
The output was:
0
Microsoft (R) Build Engine version 16.7.0+b89cb5fde for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Build started 17/10/2020 14:27:04.
Project "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj" on node 1 (default targets).
PrepareForBuild:
Creating directory "Debug\".
Creating directory "Debug\CompilerIdCUDA.tlog\".
InitializeBuildStatus:
Creating "Debug\CompilerIdCUDA.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
AddCudaCompileDeps:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe /E /nologo /showIncludes /TP /D__CUDACC__ /D_MBCS /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin" /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" /I. /FIcuda_runtime.h /c C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CMakeCUDACompilerId.cu
Project "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj" (1) is building "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj" (1:2) on node 1 (CudaBuildCore target(s)).
CudaBuildCore:
Compiling CUDA source file CMakeCUDACompilerId.cu...
cmd.exe /C "C:\Users\ekin4\AppData\Local\Temp\tmp48c6c6d6a54f48e18812e98575ede30b.cmd"
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe" -gencode=arch=compute_52,code=\"sm_52,compute_52\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -G --keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static -v -g -D_MBCS -Xcompiler "/EHsc /W0 /nologo /Od /FdDebug\vc142.pdb /FS /Zi /RTC1 /MDd " -o Debug\CMakeCUDACompilerId.cu.obj "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CMakeCUDACompilerId.cu"
C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe" -gencode=arch=compute_52,code=\"sm_52,compute_52\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -G --keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static -v -g -D_MBCS -Xcompiler "/EHsc /W0 /nologo /Od /FdDebug\vc142.pdb /FS /Zi /RTC1 /MDd " -o Debug\CMakeCUDACompilerId.cu.obj "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CMakeCUDACompilerId.cu"
#$ _NVVM_BRANCH_=nvvm
#$ _SPACE_=
#$ _CUDART_=cudart
#$ _HERE_=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin
#$ _THERE_=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin
#$ _TARGET_SIZE_=
#$ _TARGET_DIR_=
#$ _TARGET_SIZE_=64
#$ _WIN_PLATFORM_=x64
#$ TOP=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/..
#$ NVVMIR_LIBRARY_DIR=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../nvvm/libdevice
#$ PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../nvvm/bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../lib;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64;;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x86;;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\tools;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\ide;C:\Program Files (x86)\HTML Help Workshop;;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\;C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin;C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\;;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\libnvvp;C:\Program Files (x86)\Razer Chroma SDK\bin;C:\Program Files\Razer Chroma SDK\bin;C:\Program Files (x86)\Razer\ChromaBroadcast\bin;C:\Program Files\Razer\ChromaBroadcast\bin;c:\program files (x86)\common files\oracle\java\javapath;c:\program files (x86)\common files\intel\shared libraries\redist\intel64\compiler;c:\program files (x86)\razer chroma sdk\bin;c:\program files\razer chroma sdk\bin;c:\program files (x86)\razer\chromabroadcast\bin;c:\program files\razer\chromabroadcast\bin;c:\windows\system32;c:\windows;c:\windows\system32\wbem;c:\windows\system32\windowspowershell\v1.0\;c:\windows\system32\openssh\;c:\program files (x86)\intel\intel(r) management engine components\dal;c:\program files\intel\intel(r) management engine components\dal;c:\users\administrator\appdata\local\microsoft\windowsapps;c:\windows\system32\config\systemprofile\appdata\local\microsoft\windowsapps;c:\programdata\chocolatey\bin;c:\program files\git\cmd;c:\program files\microsoft sql server\client sdk\odbc\110\tools\binn\;c:\program files (x86)\microsoft sql server\120\tools\binn\;c:\program files\microsoft sql server\120\tools\binn\;c:\program files\microsoft sql server\120\dts\binn\;c:\program files (x86)\windows kits\8.1\windows performance toolkit\;c:\program files\miktex 2.9\miktex\bin\x64\;c:\program files\calibre2\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\LyX 2.3\Perl\bin;C:\WINDOWS\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Git\cmd;C:\Program Files\NVIDIA Corporation\Nsight Compute 2020.2.0\;C:\Program Files\dotnet\;C:\Users\ekin4\AppData\Local\Microsoft\WindowsApps;C:\Users\ekin4\AppData\Local\Pandoc\;C:\Users\ekin4\.dotnet\tools;
#$ INCLUDES="-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../include"
#$ LIBRARIES= "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../lib/x64"
#$ CUDAFE_FLAGS=--sdk_dir "C:\Program Files (x86)\Windows Kits\10"
#$ PTXAS_FLAGS=
CMakeCUDACompilerId.cu
#$ cl.exe @"C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-9.res" > "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-8_CMakeCUDACompilerId.cpp1.ii"
CMakeCUDACompilerId.cu
#$ erase C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-9.res
#$ cicc --microsoft_version=1927 --msvc_target_version=1927 --compiler_bindir "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/HostX64/x64/../../../../../../.." --orig_src_file_name "C:/Users/ekin4/Documents/git_repos/mitsuba_renderer_folder/mitsuba2/build/CMakeFiles/3.18.4/CompilerIdCUDA/CMakeCUDACompilerId.cu" --allow_managed --debug_mode --sdk_dir "C:\Program Files (x86)\Windows Kits\10" -arch compute_52 -m64 -ftz=0 -prec_div=1 -prec_sqrt=1 -fmad=1 --include_file_name "tmpxft_00005d50_00000000-4_CMakeCUDACompilerId.fatbin.c" -g -O0 -tused -nvvmir-library "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../nvvm/libdevice/libdevice.10.bc" --gen_module_id_file --module_id_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-5_CMakeCUDACompilerId.module_id" --gen_c_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.c" --stub_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.stub.c" --gen_device_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.gpu" "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-8_CMakeCUDACompilerId.cpp1.ii" -o "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.ptx"
#$ ptxas -arch=sm_52 -m64 -g --dont-merge-basicblocks --return-at-end "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.ptx" -o "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-10_CMakeCUDACompilerId.sm_52.cubin"
#$ fatbinary -64 --ident="C:/Users/ekin4/Documents/git_repos/mitsuba_renderer_folder/mitsuba2/build/CMakeFiles/3.18.4/CompilerIdCUDA/CMakeCUDACompilerId.cu" --cmdline="-g --dont-merge-basicblocks --return-at-end " --cicc-cmdline="-ftz=0 -prec_div=1 -prec_sqrt=1 -fmad=1 " -g "--image3=kind=elf,sm=52,file=C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-10_CMakeCUDACompilerId.sm_52.cubin" "--image3=kind=ptx,sm=52,file=C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.ptx" --embedded-fatbin="C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-4_CMakeCUDACompilerId.fatbin.c"
#$ erase C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-4_CMakeCUDACompilerId.fatbin
#$ cl.exe @"C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-11.res" > "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-6_CMakeCUDACompilerId.cpp4.ii"
CMakeCUDACompilerId.cu
#$ erase C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-11.res
#$ cudafe++ --microsoft_version=1927 --msvc_target_version=1927 --compiler_bindir "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/HostX64/x64/../../../../../../.." --orig_src_file_name "C:/Users/ekin4/Documents/git_repos/mitsuba_renderer_folder/mitsuba2/build/CMakeFiles/3.18.4/CompilerIdCUDA/CMakeCUDACompilerId.cu" --allow_managed --debug_mode --sdk_dir "C:\Program Files (x86)\Windows Kits\10" --m64 --parse_templates --gen_c_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name "tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.stub.c" --module_id_file_name "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-5_CMakeCUDACompilerId.module_id" "C:/Users/ekin4/AppData/Local/Temp/tmpxft_00005d50_00000000-6_CMakeCUDACompilerId.cpp4.ii"
#$ cl.exe @"C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-12.res" -Fo"Debug/CMakeCUDACompilerId.cu.obj"
tmpxft_00005d50_00000000-7_CMakeCUDACompilerId.cudafe1.cpp
#$ erase C:\Users\ekin4\AppData\Local\Temp/tmpxft_00005d50_00000000-12.res
Done Building Project "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj" (CudaBuildCore target(s)).
Link:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdCUDA.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib\x64" cudart_static.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdCUDA.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdCUDA.lib" /MACHINE:X64 Debug\CMakeCUDACompilerId.cu.obj
Creating library .\CompilerIdCUDA.lib and object .\CompilerIdCUDA.exp
CompilerIdCUDA.vcxproj -> C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\.\CompilerIdCUDA.exe
PostBuildEvent:
echo CMAKE_CUDA_COMPILER=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe
:VCEnd
CMAKE_CUDA_COMPILER=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe
FinalizeBuildStatus:
Deleting file "Debug\CompilerIdCUDA.tlog\unsuccessfulbuild".
Touching "Debug\CompilerIdCUDA.tlog\CompilerIdCUDA.lastbuildstate".
Done Building Project "C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CompilerIdCUDA.vcxproj" (default targets).
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:02.53
Compilation of the CUDA compiler identification source "CMakeCUDACompilerId.cu" produced "CompilerIdCUDA.exe"
Compilation of the CUDA compiler identification source "CMakeCUDACompilerId.cu" produced "CompilerIdCUDA.exp"
Compilation of the CUDA compiler identification source "CMakeCUDACompilerId.cu" produced "CompilerIdCUDA.lib"
Compilation of the CUDA compiler identification source "CMakeCUDACompilerId.cu" produced "CompilerIdCUDA.vcxproj"
The CUDA compiler identification is NVIDIA, found in "C:/Users/ekin4/Documents/git_repos/mitsuba_renderer_folder/mitsuba2/build/CMakeFiles/3.18.4/CompilerIdCUDA/CompilerIdCUDA.exe"
Parsed CUDA nvcc include information from above output:
found 'INCLUDES=' string: ["-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin/../include" ]
Where the first section
Compiling the CUDA compiler identification source file "CMakeCUDACompilerId.cu" succeeded.
Compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/bin/nvcc.exe
Build flags: ;-gencode=arch=compute_75,code=compute_75
Id flags: --cuda-gpu-arch=sm_75 -v
suggests that the compilation will be attempted with --cuda-gpu-arch=sm_75
, but in the actual compilation output we see that the compiler gets the -gencode=arch=compute_52,code=\"sm_52,compute_52\"
architecture flags in line 3:
Compiling CUDA source file CMakeCUDACompilerId.cu...
cmd.exe /C "C:\Users\ekin4\AppData\Local\Temp\tmp48c6c6d6a54f48e18812e98575ede30b.cmd"
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe" -gencode=arch=compute_52,code=\"sm_52,compute_52\" --use-local-env -ccbin \
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64" -x cu \
-I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -G \
--keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static -v -g -D_MBCS -Xcompiler "/EHsc /W0 /nologo /Od /FdDebug\vc142.pdb /FS /Zi /RTC1 /MDd " -o Debug\CMakeCUDACompilerId.cu.obj \
"C:\Users\ekin4\Documents\git_repos\mitsuba_renderer_folder\mitsuba2\build\CMakeFiles\3.18.4\CompilerIdCUDA\CMakeCUDACompilerId.cu"
There are no other architecture specifications in this call to the compiler and I cannot identify another point in the CMake CUDA detection pipeline that would lead to a failure to detect 75
and 50
as valid architectures. In fact, the only architecture that is accepted is 52
which is the default tested in CMakeDetermineCompilerId.cmake
.
If I am understanding this incorrectly or misusing the CMAKE_CUDA_ARCHITECTURES
flag, I’d be curious to know exactly how I should specify the architecture.