jpfeuffer
(Julianus Pfeuffer)
March 18, 2022, 10:42am
1
Hi,
Did anyone successfully use the FindPython module on manylinux?
I tried -DPython_ROOT_DIR and -DPython_EXECUTABLE to set to /opt/python/cp39-cp39/bin/python or the respective folder.
If I print the found python Version it says 2.7.
-- Could NOT find Python (missing: Interpreter Development Development.Module Development.Embed) (found version "2.7")
And Python_add_library does not exist although I am using CMake 3.23. Is the existence depending on the found Python version/components?
Thanks
Python_add_library
is defined only if usable (i.e. at least, one of Development components if found).
What is your CMake version? Can-you provide an example showing the problem?
jpfeuffer
(Julianus Pfeuffer)
March 18, 2022, 4:27pm
3
Thanks a lot for your quick reply. That makes sense. I think I found the problem! On manylinux you have to reduce to Development.Module.
opened 02:49PM - 24 May 20 UTC
closed 01:18PM - 25 May 20 UTC
<sup>Please let me know if this is better directed towards CMake itself.</sup>
…
I am attempting to build manylinux2014 wheels for [pytype](https://github.com/google/pytype), which uses CMake and its [FindPythonLibs](https://cmake.org/cmake/help/latest/module/FindPythonLibs.html) function. It is having trouble finding `libpythonX.Y.so` in certain cases.
Putting pytype aside, here is an MVCE:
Container shell:
```
docker container run -it --rm --workdir='/foo' \
quay.io/pypa/manylinux2014_x86_64:latest
```
Example:
```bash
yum install -y cmake ninja-build python3-devel
{
echo 'project(Foo)'
echo 'cmake_minimum_required(VERSION 2.8)'
echo 'find_package(PythonLibs)'
echo 'if(NOT PYTHONLIBS_FOUND)'
echo ' message(FATAL_ERROR "Could not find developer libraries")'
echo 'endif()'
} > CMakeLists.txt
cmake -G Ninja -DPython_ADDITIONAL_VERSIONS=3.7
```
Output:
```
-- The C compiler identification is GNU 9.1.1
-- The CXX compiler identification is GNU 9.1.1
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Ninja
-- Check for working CXX compiler using: Ninja -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
CMake Error at CMakeLists.txt:5 (message):
Could not find developer libraries
-- Configuring incomplete, errors occurred!
See also "/foo/CMakeFiles/CMakeOutput.log".
```
Here the `-DPython_ADDITIONAL_VERSIONS=3.7` comes from (I think) invoking the pytype `bdist_wheel` with `/opt/python/cp37-cp37m/bin/python`, for example.
Looking more closely, I see:
```
$ ls -1 /usr/lib64/libpython*
/usr/lib64/libpython2.7.so.1.0
/usr/lib64/libpython3.6m.so
/usr/lib64/libpython3.6m.so.1.0
/usr/lib64/libpython3.so
```
where the final 3 entries have been put there via the `python3-devel` install. I also see that the `python3-devel` install fetched `python3-libs-3.6.8-13.el7.x86_64.rpm` specifically.
So it looks like [`cmake` is looking for a MAJOR.MINOR version](https://github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake#L84), rather than just `libpython3.so`.
And indeed, if I run:
```
alias clean='rm -rvf CMakeCache.txt CMakeFiles'
clean
cmake -G Ninja -DPython_ADDITIONAL_VERSIONS=3.6
```
Then the cmake invocation succeeds:
```
-- The C compiler identification is GNU 9.1.1
-- The CXX compiler identification is GNU 9.1.1
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Ninja
-- Check for working CXX compiler using: Ninja -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found PythonLibs: /usr/lib64/libpython3.6m.so (found version "3.6.8")
-- Configuring done
-- Generating done
-- Build files have been written to: /foo
```
How can I allow FindPythonLibs to succeed here? Does cmake need the `libpython3.so` or a specific MAJOR.MINOR version? Would I need to install something like `python37-devel`? (I don't see it on the default yum repos so would probably need to get it from somewhere else.)
I just hoped for a more intuitive error message. Especially since the Interpreter was definitely found.
And it just ignored the specified executable and went searching for something else.