# Correct use of CMAKE\_CUDA\_ARCHITECTURES
**URL:** https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250
**Category:** Code
**Tags:** lang:cuda
**Created:** [May 21, 2020, 8:49pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250 "2020-05-21T20:49:45Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@mojca](https://discourse.cmake.org/u/mojca)
#### Post date: [May 21, 2020, 8:49pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/1 "2020-05-21T20:49:45Z")
I was looking for ways to properly target different compute capabilities of cuda devices and found a couple of new policies for 3.18. So I tried (simplified):
```auto
cmake_minimum_required(VERSION 3.17 FATAL_ERROR)
cmake_policy(SET CMP0104 NEW)
cmake_policy(SET CMP0105 NEW)
...
add_library(hello SHARED hello.cpp hello.h hello.cu)
set_property(TARGET hello PROPERTY CUDA_ARCHITECTURES 52 61 75)
```
During configuration I have tried to set `CMAKE_CUDA_ARCHITECTURES` to `52 61 75` in the GUI, but the compiler test fails with
```auto
Compiling CUDA source file main.cu...
C:\MyProject\build\vs2019-x64\CMakeFiles\CMakeTmp>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\nvcc.exe" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.27.28826\bin\HostX64\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include" --keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static 61 75,code=[compute_52 61 75,sm_52 61 75] -Xcompiler="/EHsc -Zi -Ob0" -g -D_WINDOWS -D"CMAKE_INTDIR=\"Debug\"" -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /FdcmTC_e9572.dir\Debug\vc142.pdb /FS /Zi /RTC1 /MDd /GR" -o cmTC_e9572.dir\Debug\main.obj "C:\MyProject\build\vs2019-x64\CMakeFiles\CMakeTmp\main.cu"
nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
```
Note the strange flag `--compile -cudart static 61 75,code=[compute_52 61 75,sm_52 61 75]`
I installed CMake 3.17.20200520-g81e8f62 on Windows, I’m using Visual Studio 10.19 (16.7.0 Preview 1.0) with NVidia’s SDK 10.2.
---
### Author:  [@marc.chevrier](https://discourse.cmake.org/u/marc.chevrier)
#### Post date: [May 21, 2020, 9:15pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/2 "2020-05-21T21:15:44Z")
For a correct behavior, you need on-going version 3.18 but requested version3.17.
Try following syntax: `cmake_minimum_required(VERSION 3.17...3.18)` to get expected behavior regarding new features introduced for future v3.18.
---
### Author:  [@mojca](https://discourse.cmake.org/u/mojca)
#### Post date: [June 12, 2020, 1:44pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/3 "2020-06-12T13:44:55Z")
I could not make this work until I switched to CMake 3.18 (rc1).
I wasn’t sure that my VS/CUDA setup wasn’t broken until the upgrade of CMake solved the problem.
But I’m experiencing a strange bug now. I’m using
```cmake
if(NOT DEFINED ${CMAKE_CUDA_ARCHITECTURES})
set(CMAKE_CUDA_ARCHITECTURES 52 61 75)
endif()
message(STATUS "CUDA architectures set to ${CMAKE_CUDA_ARCHITECTURES}")
```
which prints out
```auto
CUDA architectures set to 52;61;75
```
and the compile flag seem to be correct. However the cmake-gui still shows `CMAKE_CUDA_ARCHITECTURES` being set to 30.
---
### Author:  [@marc.chevrier](https://discourse.cmake.org/u/marc.chevrier)
#### Post date: [June 12, 2020, 3:11pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/4 "2020-06-12T15:11:06Z")
`cmake-gui` display variables defined in the [cache](https://cmake.org/cmake/help/latest/manual/cmake-language.7.html#variables).
The command `set(CMAKE_CUDA_ARCHITECTURES 52 61 75)` defines standard variable which hide the cache variable but do not overwrite it. If you want to overwrite the cache variable, you have to edit it with `cmake-gui` or using the following syntax: `set( ... CACHE FORCE)`, see [set](https://cmake.org/cmake/help/git-master/command/set.html) command.
---
### Author:  [@mojca](https://discourse.cmake.org/u/mojca)
#### Post date: [June 23, 2020, 12:14pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/5 "2020-06-23T12:14:08Z")
Thank you. This only solves half of the problem though.
Using
```auto
set(CMAKE_CUDA_ARCHITECTURES 52 60 61 75 CACHE STRING "CUDA architectures" FORCE)
```
works perfectly well to set some reasonable defaults. However it sadly prevents anyone from overriding these values.
For `CMAKE_INSTALL_PREFIX` I’m using the following trick:
```cmake
if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
set(CMAKE_INSTALL_PREFIX "/some/path" CACHE PATH "Default location for installing the library" FORCE)
endif(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
```
but I cannot figure out how to do something similar for CUDA architectures. All I want is basically just replace the default defaults of 30 to something more reasonable for our project, but still give the user building the binaries some freedom.
---
### Author:  [@marc.chevrier](https://discourse.cmake.org/u/marc.chevrier)
#### Post date: [June 23, 2020, 3:17pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/6 "2020-06-23T15:17:53Z")
Not at all. When, in a `CMakeLists.txt`, the variable is set (without `CACHE` specification), this variable takes precedence over the cache definition.
For example
```cmake
# First definition in the cache
set(CMAKE_CUDA_ARCHITECTURES 52 60 61 75 CACHE STRING "CUDA architectures")
# ... after in some CMakeLists.txt
set (CMAKE_CUDA_ARCHITECTURES 52 60 70 90)
message ("CUDA ARCHS: ${CMAKE_CUDA_ARCHITECTURES}")
```
You get value `CUDA ARCHS: 52;60;70;90` displayed, not the value defined in the cache.
The idea behind the cache is to ensure some defaults are available each time `cmake` is relaunched without preventing the user to override these defaults.
So the cache value is defined on the command line or with `ccmake` or `ccmake-gui` and after, in the `CMakeLists.txt` files, it is possible to override this default easily by just setting the variable (`set` command without `CACHE` option) without changing the cache value.
---
### Author:  [@mojca](https://discourse.cmake.org/u/mojca)
#### Post date: [June 23, 2020, 8:06pm UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/7 "2020-06-23T20:06:50Z")
Thank you for the explanation, it clarified some aspects of how CMake handles cache, so I can better understand what is going on under the hood, but I still don’t quite get it how to achieve what I’m striving for.
For simplicity let’s assume that we only have a trivial hello-world library with a single `CMakeLists.txt`, so there is no need for different `CMakeLists.txt` inside the project to override each other. (Still, I don’t want to specify this property on per-target basis; I just want to be able to specify the list of architectures at a single place and keep it consistent across the project.) I would like to support the following two scenarios:
Case 1: The user compiling the library doesn’t do anything extra (no extra options are specified). I would want to compile support for architectures “`52 60 61 75`” in that case.
Case 2: The user compiling the library runs `cmake-gui`, selects `Configure`. The value of `CMAKE_CUDA_ARCHITECTURES` should read `52;60;61;75` (not `30`). The user changes that value to `70`, clicks `Configure`, then `Generate`. The option `70` stays (it should not be automatically changed back to `52;60;61;75`) and the generated project has the correct flags just for architecture `70`.
The code
```cmake
set(CMAKE_CUDA_ARCHITECTURES 52 60 61 75)
```
is not really helpful. It does work the way you describe it, it does have the desired effect of enforcing “my favourite” list of architectures, but first of all it lies to the user about the architectures being used (the GUI claims that the architecture `30` would be used) which is not too helpful, and even more importantly it prevents the user from having any effect on the set of architectures being used for the actual compilation. (It does keep the user selection shown, but that selection is ignored.)
By using
```cmake
set(CMAKE_CUDA_ARCHITECTURES 52 60 61 75 CACHE STRING "CUDA architectures" FORCE)
```
on the other hand the `cmake-gui` correctly reports the set of selected architectures to the user. But there doesn’t seem to be any way to change that selection. Whenever the user changes the value and clicks `Configure` or `Generate`, the values are reset back to `52;60;61;75`.
Maybe something along the lines of the following code could have worked (or at least something similar works for setting the default prefix):
```cmake
if(CMAKE_CUDA_ARCHITECTURES_INITIALIZED_TO_DEFAULT)
set(CMAKE_CUDA_ARCHITECTURES 52 60 61 75 CACHE STRING "CUDA architectures" FORCE)
endif(CMAKE_CUDA_ARCHITECTURES_INITIALIZED_TO_DEFAULT)
```
except that no such function (`CMAKE_CUDA_ARCHITECTURES_INITIALIZED_TO_DEFAULT`) exists yet, and introducing one probably doesn’t scale either.
---
### Author:  [@MathiasMagnus](https://discourse.cmake.org/u/MathiasMagnus)
#### Post date: [October 14, 2020, 8:32am UTC](https://discourse.cmake.org/t/correct-use-of-cmake-cuda-architectures/1250/8 "2020-10-14T08:32:22Z")
I don’t really see the value of hardcoding binary formats into a build script, since this list is likely a user preference depending on what machines they wish to target with their build. I tried supplying `CMAKE_CUDA_ARCHITECTURES` from the command-line, passing two parameters, but it never works. With the value that’s being passed being the literal `52 70` or `52;70` neither work and results in nvcc compiler check failures.
```auto
[cmake] -- Check for working CUDA compiler: /usr/local/cuda-10.2/bin/nvcc - broken
[cmake] CMake Error at /snap/cmake/619/share/cmake-3.18/Modules/CMakeTestCUDACompiler.cmake:52 (message):
[cmake] The CUDA compiler
[cmake]
[cmake] "/usr/local/cuda-10.2/bin/nvcc"
[cmake]
[cmake] is not able to compile a simple test program.
[cmake]
[cmake] It fails with the following output:
[cmake]
[cmake] Change Dir: /home/mate/Source/Repos/gromacs/src/benchmark_amd/.vscode/build/CMakeFiles/CMakeTmp
[cmake]
[cmake] Run Build Command(s):/home/mate/.local/bin/ninja cmTC_dc8b7 && [1/2] Building CUDA object CMakeFiles/cmTC_dc8b7.dir/main.cu.o
[cmake] FAILED: CMakeFiles/cmTC_dc8b7.dir/main.cu.o
[cmake] /usr/local/cuda-10.2/bin/nvcc -forward-unknown-to-host-compiler --generate-code=arch=compute_52;70,code=[compute_52;70,sm_52;70] -MD -MT CMakeFiles/cmTC_dc8b7.dir/main.cu.o -MF CMakeFiles/cmTC_dc8b7.dir/main.cu.o.d -x cu -c main.cu -o CMakeFiles/cmTC_dc8b7.dir/main.cu.o
[cmake] nvcc fatal : Option '--generate-code arch=compute_52', missing code
[cmake] /bin/sh: 1: 70,code=[compute_52: not found
[cmake] /bin/sh: 1: 70,sm_52: not found
[cmake] /bin/sh: 1: 70]: not found
[cmake] ninja: build stopped: subcommand failed.
```
The command assembled by CMake is garbage in different ways depending on whether there’s a space or a semi-colon inside the command. The variable when taken from the command-line seems to be taken verbatim and isn’t interpreted as a CMake list.
_(Note that I omit the details of how many quotes and escapes needed. I’m parametrizing the CMake invocation from VS Code via CMake Tools. I’m checking the actual command-line invocation and CMake Tools seems to correctly quote/escape/massage command-line args when containing apostrophes, backslashes, semi-colons, etc. I experience all this using `cmake version 3.18.4`)_