Python Interpreter found but Python_EXECUTABLE not populated

Excerpt from my CMakeLists.txt

add_custom_command(
    OUTPUT ${PROJECT_SOURCE_DIR}/tests/bit_tests_lut.c
    COMMAND ${Python_EXECUTABLE} ${PROJECT_SOURCE_DIR}/python/generate_bit_lut_for_unit_tests.py
    DEPENDS ${PROJECT_SOURCE_DIR}/python/generate_bit_lut_for_unit_tests.py
)
add_custom_target(
    generated_bit_tests_lut
    DEPENDS ${PROJECT_SOURCE_DIR}/tests/bit_tests_lut.c
)

Output

-- The C compiler identification is GNU 11.1.0
-- The CXX compiler identification is GNU 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: /usr/bin/python3.9 (found version "3.9.5") found components: Interpreter 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/satrapes/workspace/satrapes_bit_lib/cmake_bin
[  7%] Generating ../../tests/bit_tests_lut.c
/bin/sh: line 1: ../../python/generate_bit_lut_for_unit_tests.py: Permission denied

So as you can see python interpreter executable has been found but it has not populated the relevant variable. (You notice that by the absence of python or /usr/bin/python3.9 after /bin/sh: line 1:

I have also tried ${Python2_EXECUTABLE}, ${Python3_EXECUTABLE} and ${PYTHON_EXECUTABLE}

Any suggestions please?

Can you show an SSCCE of your CMake code? Namely, does the find_package() call come before the add_custom_command(), and is it in correct scope?

You are right about the sscce, I will show it later today when I get home if needed.

As far as I understand CMake does its introspection thing and detects python by itself. I don’t explicitly call find_package(). I assume that my commands run after that phase where I would have expected Python_EXECUTABLE to have been populated since it says it has found the Python Interpreter.

What I can tell you from memory is that the add_custom_command is in root_dir/tests/CMakeLists.txt and there is a line in root_dir/CMakeLists.txt that would add_subdirectory(tests). The root CMakeLists.txt doesn’t do much apart from defining version and adding subdirectories.

Not unless someone tells it to do so (your project, something you find_package for, your toolchain file(?)). A minimal CMake project does not look for Python. Locate who’s asking for it, and you may find something to help you solve the issue. CMake’s --debug-find command-line option might come in handy.

1 Like

CMake messages show that Python is effectively founded so find_package(Python) is called somewhere and Python_EXECUTABLE is defined. But, clearly, not in the scope (surely in some lower scope) where add_custom_command is used. This explain why Python_EXECUTABLE is not defined.

1 Like

Thank you very much @Angew @marc.chevrier.
You were right in everything and I am sorry that I haven’t replied earlier but it got busy at work and didn’t have any time.
I was adding the subdirectory for googletests which looked for the python executable, prior to the custom command. My project is structured according to this site GoogleTest · Modern CMake (the submodule method).
So I think that it has to do with the scope (which I only now realized partly how it works). So I guess that Python_EXECUTABLE is set in the scope of the add_subdirectory(googletest) command and that is why it is not seen in the following command. Any thoughts on how to handle this?
I have thought about maybe doing an include instead of an add_subdirectory and the other option is to do another find_package(Python) in the parent scope.
I don’t like the second choice much because you duplicate the effort.

Just add find_package(Python...) command where you need python! This is the normal way to use packages.

There is no real duplication because most of the costly work is stored in cached variables so subsequent calls of find_package(Python..) had nearly no cost.

2 Likes

Thanks @marc.chevrier