find_package(Python3) is not finding the correct python

Code
1

I have the following minimal CMakeLists.txt:

cmake_minimum_required (VERSION 3.8)
project (myproject C CXX)
find_package (Python3 COMPONENTS Interpreter Development REQUIRED)

I use spack (https://spack.io/) to manage my dependencies, so I have a spack environment activated which contains a build of python 3.11.7, that is, this version of python is in my $PATH, python --version gives me “3.11.7”, $PYTHONPATH contains the correct path for this python build, and $LD_LIBRARY_PATH contains the path where libpython3.11.so can be found (and it does not contain “/usr/lib” or any other system path).

When I call cmake, however, I’m getting this error:

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Development
  Development.Module Development.Embed) (found version "3.10.12")
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.22/Modules/FindPython/Support.cmake:3180 (find_package_handle_standard_args)
  /usr/share/cmake-3.22/Modules/FindPython3.cmake:490 (include)
  CMakeLists.txt:3 (find_package)

The found version "3.10.12" makes me think that cmake is finding the version of python that is in “/usr/bin”, even though “/usr/bin” comes much later in my $PATH than the path to the python version I’m trying to use. The system-provided python doesn’t have the development libraries, which is why cmake fails, I believe.

Could you help me figure out why FindPython3.cmake finds this system-provided Python instead of the one I want to use, and how to fix this problem?

2

What names are installed for python from spack? Is python3 or python3.11 available from spack?
Can you try to set the variable Python3_FIND_STRATEGY to value LOCATION? From your snippet, the behavior for the find strategy is VERSION so can get is the observed result if spack do not install versioned name (.i.e. python3.11).

3

Setting Python3_FIND_STRATEGY to LOCATION works. Is that the solution to go for, moving forward? I still don’t understand why cmake picks 3.10 when the only variable that references it is $PATH, but “/usr/bin” is at the end of it…

Spack does install python, python3, and python3.11. Doing which pythonX for these pythons executables all yields the spack-installed ones.

Something I suspected to happen is that in /usr/share/cmake-3.22/Modules/FindPython/Support.cmake, near the beginning of the file, I see this:

set(_${_PYTHON_PREFIX}_VERSIONS 3.10 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0)

I added 3.11 in the list, but it didn’t change anything.

4

The fact that version 3.11 is not defined is the root of the problem.
Anyway, to get your expected behavior, I recommend you to set variable Python3_FIND_STRATEGY with value LOCATION or set policy CMP0094 to NEW.

By the way, Python3 version 3.11 was added in CMake version 3.22.2.

To finish, I am surprised that adding 3.11 didn’t change anything. Did you clean-up the environment (i.e. at least removing CMakeCache.txt file) before you try?

5

Yes, I cleaned up everything (I reconfirmed just now).