cmake_minimum_required (VERSION 3.8)
project (myproject C CXX)
find_package (Python3 COMPONENTS Interpreter Development REQUIRED)
I use spack (https://spack.io/) to manage my dependencies, so I have a spack environment activated which contains a build of python 3.11.7, that is, this version of python is in my $PATH, python --version gives me “3.11.7”, $PYTHONPATH contains the correct path for this python build, and $LD_LIBRARY_PATH contains the path where libpython3.11.so can be found (and it does not contain “/usr/lib” or any other system path).
When I call cmake, however, I’m getting this error:
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Development
Development.Module Development.Embed) (found version "3.10.12")
Call Stack (most recent call first):
/usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-3.22/Modules/FindPython/Support.cmake:3180 (find_package_handle_standard_args)
/usr/share/cmake-3.22/Modules/FindPython3.cmake:490 (include)
CMakeLists.txt:3 (find_package)
The found version "3.10.12" makes me think that cmake is finding the version of python that is in “/usr/bin”, even though “/usr/bin” comes much later in my $PATH than the path to the python version I’m trying to use. The system-provided python doesn’t have the development libraries, which is why cmake fails, I believe.
Could you help me figure out why FindPython3.cmake finds this system-provided Python instead of the one I want to use, and how to fix this problem?
What names are installed for python from spack? Is python3 or python3.11 available from spack?
Can you try to set the variable Python3_FIND_STRATEGY to value LOCATION? From your snippet, the behavior for the find strategy is VERSION so can get is the observed result if spack do not install versioned name (.i.e. python3.11).
Setting Python3_FIND_STRATEGY to LOCATION works. Is that the solution to go for, moving forward? I still don’t understand why cmake picks 3.10 when the only variable that references it is $PATH, but “/usr/bin” is at the end of it…
Spack does install python, python3, and python3.11. Doing which pythonX for these pythons executables all yields the spack-installed ones.
Something I suspected to happen is that in /usr/share/cmake-3.22/Modules/FindPython/Support.cmake, near the beginning of the file, I see this:
The fact that version 3.11 is not defined is the root of the problem.
Anyway, to get your expected behavior, I recommend you to set variable Python3_FIND_STRATEGY with value LOCATION or set policy CMP0094 to NEW.
By the way, Python3 version 3.11 was added in CMake version 3.22.2.
To finish, I am surprised that adding 3.11 didn’t change anything. Did you clean-up the environment (i.e. at least removing CMakeCache.txt file) before you try?
1, It seems to me that FindPython should (by default) pick up the version correspoding to the python3 command.
2. I cannot find the documentation or an example for Python3_FIND_STRATEGY so I do know how to use it to get FindPython to get version 3.12 of python.
cmake_minimum_required (VERSION 3.8)
project (myproject C CXX)
set( Python_FIND_UNVERSIONED_NAMES FIRST )
find_package (Python3 COMPONENTS Interpreter Development REQUIRED )
CMake output:
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /usr/bin/python3.13 (found version "3.13.0") found components: Interpreter Development Development.Module Development.Embed
-- Configuring done (0.5s)
-- Generating done (0.0s)
-- Build files have been written to: /home/bradbell/trash/cmake
You don’t get the expected result for t0i reasons:
You have changed the module used between the examples (Python and Python3). Variable’s prefix must be used accordingly with the module name.
Because you request version 3.8 for CMake.
In this case, Python3_FIND_STRATEGY variable is set to VERSION which means that, for each possible python version (starting from the most recent), all locations are searched for it so it is normal that version 3.13 is returned.
By setting Python3_FIND_STRATEGY variable to LOCATION, all possible versions, ordered from the most recent to the least one, will be searched in all locations. In this case, by setting Python3_FIND_UNVERSIONED_NAMES variable to FIRST, the unversioned python will be selected.
Thanks. I thought that the minimum requires cmake version would only check that your version was recent enough. I did not realize that it actually changes how ones version of cmake behaves.
CMakeLists.txt
# Must use 3.15 or higher in order to find python3.12 instead of python3.13
cmake_minimum_required (VERSION 3.15)
project (myproject C CXX)
set( Python_FIND_UNVERSIONED_NAMES FIRST )
find_package (Python COMPONENTS Interpreter Development REQUIRED )
Results in
-- Found Python: /usr/bin/python3 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed
We solve this by other ways…
we require the user to have an ENV var that points to the PythonEXE to use.
ie: COMPANY_PYTHON3_EXE=/absolute/path/to/python.
This may be a shell script that “sanitizes” the ENV variables.
Why? Because some tools {MicroSemi} install a private copy of Python as part of their eclipse install and then they set various PYTHON VARS like PYTHON_HOME to point to their directories.
And for example PY VIRT env does not work because - a) it really loads another sub-shell, and well - then you run python in that sub-shell… and (b) Some tools [xilinx] reset your PATH to have the standard path… and other things… Besides using something that creates a subshell - means you cannot use your desktop icons to launch tools and have it work.
So - instead - we use (a) env variable to the executable (b) that executable [script] must sanitize the ENV vars before launching python
In CMake - we read the ENV var and set the CMAKE variable for the python exe