Hi,
The issue I bumped into just now may be a very corner case thing, but I think I found a bug…
One of the environments in which I’d like to build CUDA code is one where host code is being compiled with a version of clang, which itself was not built against the system’s default installation of GCC, but rather a custom GCC installation. I have the following set up:
CXX="/software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++"
CXXFLAGS="--gcc-toolchain=/software/gcc/13.3.0/x86_64-el9"
CUDACXX="/software/cuda/12.8.1/x86_64/bin/nvcc"
CUDAHOSTCXX="/software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++"
CUDAFLAGS="-Xcompiler --gcc-toolchain=/software/gcc/13.3.0/x86_64-el9 -allow-unsupported-compiler"
In this environment, I try to build the following simple project:
# Set up the project.
cmake_minimum_required(VERSION 3.20)
project(CMakeCUDABug VERSION 1.0.0 LANGUAGES CXX CUDA)
# Build a simple CUDA executable.
add_executable(example source.cpp source.cu)
The contents of source.cpp and source.cu are not important, the important bit is just that I link both C++ and CUDA code into a single binary.
In the environment that I described earlier, I see the following:
[bash][atspot01]:development > cmake -S cuda_bug/ -B build
-- The CXX compiler identification is IntelLLVM 2025.1.0
-- The CUDA compiler identification is NVIDIA 12.8.93 with host compiler IntelLLVM 2025.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /software/cuda/12.8.1/x86_64/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Configuring done (2.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/krasznaa/development/build
[bash][atspot01]:development > VERBOSE=1 cmake --build build/
Change Dir: '/home/krasznaa/development/build'
Run Build Command(s): /software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile
/software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -S/home/krasznaa/development/cuda_bug -B/home/krasznaa/development/build --check-build-system CMakeFiles/Makefile.cmake 0
/software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -E cmake_progress_start /home/krasznaa/development/build/CMakeFiles /home/krasznaa/development/build//CMakeFiles/progress.marks
/usr/bin/gmake -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory '/home/krasznaa/development/build'
/usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/depend
gmake[2]: Entering directory '/home/krasznaa/development/build'
cd /home/krasznaa/development/build && /software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -E cmake_depends "Unix Makefiles" /home/krasznaa/development/cuda_bug /home/krasznaa/development/cuda_bug /home/krasznaa/development/build /home/krasznaa/development/build /home/krasznaa/development/build/CMakeFiles/example.dir/DependInfo.cmake "--color="
gmake[2]: Leaving directory '/home/krasznaa/development/build'
/usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/build
gmake[2]: Entering directory '/home/krasznaa/development/build'
[ 33%] Building CXX object CMakeFiles/example.dir/source.cpp.o
/software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++ --gcc-toolchain=/software/gcc/13.3.0/x86_64-el9 -MD -MT CMakeFiles/example.dir/source.cpp.o -MF CMakeFiles/example.dir/source.cpp.o.d -o CMakeFiles/example.dir/source.cpp.o -c /home/krasznaa/development/cuda_bug/source.cpp
[ 66%] Building CUDA object CMakeFiles/example.dir/source.cu.o
/software/cuda/12.8.1/x86_64/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++ -Xcompiler --gcc-toolchain=/software/gcc/13.3.0/x86_64-el9 -allow-unsupported-compiler -std=c++17 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -MD -MT CMakeFiles/example.dir/source.cu.o -MF CMakeFiles/example.dir/source.cu.o.d -x cu -c /home/krasznaa/development/cuda_bug/source.cu -o CMakeFiles/example.dir/source.cu.o
nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
[100%] Linking CXX executable example
/software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -E cmake_link_script CMakeFiles/example.dir/link.txt --verbose=1
/software/intel/oneapi-2025.1.0/compiler/2025.1/bin/compiler/clang++ --gcc-toolchain=/software/gcc/13.3.0/x86_64-el9 CMakeFiles/example.dir/source.cpp.o CMakeFiles/example.dir/source.cu.o -o example -L/software/cuda/12.8.1/x86_64/targets/x86_64-linux/lib/stubs -L/software/cuda/12.8.1/x86_64/targets/x86_64-linux/lib -L/usr/lib/gcc/x86_64-redhat-linux/11 -lcudadevrt -lcudart_static -lrt -lpthread -ldl
gmake[2]: Leaving directory '/home/krasznaa/development/build'
[100%] Built target example
gmake[1]: Leaving directory '/home/krasznaa/development/build'
/software/cmake/4.0.2/x86_64-el9-gcc13-opt/bin/cmake -E cmake_progress_start /home/krasznaa/development/build/CMakeFiles 0
[bash][atspot01]:development >
The issue hides very sneakily in the final link command. The compilation commands work as they should, they set up clang++ and nvcc to behave how I’d like them to. But the final link has this element that I can’t explain:
-L/usr/lib/gcc/x86_64-redhat-linux/11
And while in this piece of demonstrator code it doesn’t cause an issue, in a real project of mine it does. It leads to a link failure, as the link tries to use an incorrect version of libstdc++.so. 
I know this is a complicated build setup, but it should still be possible to make it work I hope. 
Did others bump into this in the past, is this a known issue?
Cheers,
Attila