Recommendations for Python module inclusion

I’ll just preface this by saying that most of my interactions with Python packaging has been through projects that additionally provide Python modules, but are meant to be primarily used through some other means (e.g., C++ headers and libraries). For projects that are Python modules that just happen to be written in C++, everything is probably much rosier and the above works. But it is, AFAIK, not a solution for projects which are not “Python package, but written in C++”.

I’ve been doing this for VTK, but AFAICT, sdists are just impossible to do for complex projects (namely where configuring what you want from it is not just a list of “please add this to the build” for whatever Python packaging calls the [name1,name2] suffixes). So VTK just can’t provide one. I’ve also found wheels to be terrible artifacts because they basically assume that all that matters is a project’s Python interface. This is generally not true, so they’re quite unsuitable for many projects that happen to provide Python modules in addition to some C or C++ interface. As such, VTK’s wheel is largely a dead end for any VTK-consuming projects that want something other than its Python interface[1]. Maybe someone will come up with a spec on how to ship headers, CMake package files, etc. in wheels, but I’m not holding my breath.

I don’t think CMake fits for this (though I’ve just glanced through the PEP). Maybe CPack could do it, but this is basically another “language” for CMake to handle and CMake isn’t going to link to Python anytime soon in order to read whatever configuration is needed for PEP517 compliance.

<sidenote> Python packaging is quite a mess and I don’t think I’d like to see CMake add that to its large plate of problems it has to deal with in the C and C++ world already. I wish the Python community would apply some of the Zen of Python (namely “There should be one-- and preferably only one --obvious way to do it.”) and apply it to the packaging mess that exists. Maybe CMake could then find some way to interface with it. But, that’s also basically the same problem that CMake has for C and C++ compilers today (except CMake doesn’t even have a handle on a C or C++ implementation nevermind the primary one, though that difference doesn’t make it easy either).

[1] There is apparently a hack to build against the build tree that made a particular wheel, but this is definitely a hack and not something I would call a solution.

1 Like