Idea for CMake-based package manager

I’ve been looking into Conan, but what bothers me about it is the fact that from within the project, you have to change the CMake code, to link against a CONAN_PKG::Target target.

I think the best solution is for the project’s CMake code to have no knowledge of whatever package manager the developer is using.

I’ve been thinking about how this could be achieved, and I’ve got a rough idea, so I wanted to share it in case I’m totally barking up the wrong tree here…

So, my thinking is, in every single case the project should simply call find_package (dependency). I think the job of a CMake-based package manager should be to generate find modules for use by the project. The project could provide a simple JSON or INI file that maps package names to git repositories, and then the package manager can generate for each one a find module that uses FetchContent to download the sources and add_subdirectory() to bring it into the build.

This way, system-installed packages will still be found first, and downloading/building from source will only be a fallback measure.

Obviously this approach would rely on dependencies having publicly available source code, and well-written CMakeLists.txt. Perhaps eventually this package manager could support downloading prebuilt binaries, but the basic idea would be the same: it would generate a find module that simply creates an imported target pointing at the downloaded binary, and as far as the project is concerned, it just calls find_package (Foo).

The project could still provide its own find modules for dependencies that require special handling or more complex logic.

Any thoughts or ideas about this?

You’re pulling on a very long thread. :wink: Take a look at the following issue which discusses this area more broadly:

https://gitlab.kitware.com/cmake/cmake/-/issues/22619

Be warned, it is long and goes off on multiple tangents, then eventually splits off into a set of other issues to try to capture all the different discussions. You might also want to look into CPM, which possibly does things similar to what you’ve described. What you’re describing also isn’t all that far from how vcpkg works (except none of the package managers typically prefer the system-installed packages but instead prefer to provide things themselves - which makes sense, because otherwise you can’t guarantee a consistent set of dependencies).

FWIW, I am planning on getting back to working on that area after CMake 3.23 is released.

2 Likes

Thanks for the response!

This thread is fascinating, I’ve begun reading it but you’re right that it is very long :sweat_smile: I like your idea of a PROJECT_SETUP injection file, and I also like the idea of allowing package managers to “intercept” find_package calls. My initial thought process of a package manager generating find modules for the project is a possible way to achieve essentially this same result.

I have used CPM, and I like many of its features, but what I don’t like about it is:

  • It requires you to call CPMAddPackage() instead of find_package()
  • It doesn’t support pre-built binaries

I haven’t looked very deeply at vcpkg, I’ll revisit it.

As I’ve been thinking about this, one thing I’ve pondered is the ability for CMake to print out a list of dependencies added by a project – what if there were another command package managers could run, prior to CMake configuration, something like cmake --discover-packages which would run all the configuration logic, but simply recording calls to find_package or FetchContent without actually attempting to resolve dependencies or generate the buildsystem. CMake could then output a JSON object containing the name and version of each required dependency.
Pros I see to this approach:

  • This would allow package managers to detect if a project doesn’t need all its possible dependencies for a certain configuration, for example if you’re not building tests, you don’t need to pull extra dependencies for tests
  • If a dependency is requested more than once with different versions, the package manager can resolve this or issue a diagnostic

Difficulties with this approach:

  • Projects may have conditional logic based on whether a package was found, so running the configuration logic without finding any dependencies may result in different logic than once the dependencies are populated
  • More implementation burden, to add a new command-line interface

Again, I haven’t read that whole thread you linked, but I intend to do so as I’m very interested in this topic. Apologies if I’m reiterating anything already suggested in that thread.

I’ve never contributed to CMake before, but I’m quite interested in this topic and would be eager to contribute in any way I can, if there’s a chance of any of these features getting upstreamed.

Have you heard of vcpkg?

What works for me best is CPM.cmake >GitHub - cpm-cmake/CPM.cmake: 📦 CMake's missing package manager. A small CMake script for setup-free, cross-platform, reproducible dependency management.>.
It is a wrapper around FetchContent.

/Martin

Conan is what you need, It generate Config.cmake of your dependencies (even if its build system is not cmake). And so, your project is completely independent from the package manager

HTH

David

1 Like

check this out GitHub - conan-io/cmake-conan: CMake wrapper for conan C and C++ package manager, it allows you to use find_package and link libraries as how you would do normally

Absolutely impossible today :slight_smile: . Components make this trivially unsolvable (because knowing what CMake option a given component maps to is not solvable and components are merely strings; what they actually mean differs from package to package). In addition, some projects do “try to find, set up defaults based on the presence or not”, so after resolving X, Y may show up as its dependency and be used silently as well. I far prefer the “expose an option and either use it or not based on the option” pattern (basically, every single find_package() has REQUIRED on it) because features toggling themselves based on such ambient state is maddening to debug.

1 Like

Conan doesn’t require you to change your CMake code in general but it can cause you to change how find_package and how the resulting targets are used because the resulting find modules are different. Conan can provide a toolchain that you can inject into your build during the cmake configure phase. They appear to be moving more towards that direction which has the benefit of being less intrusive.

You may try this project: GitHub - cpp-best-practices/cpp_starter_project: A template CMake project to get you started with C++ and tooling

It is prepared to use conan to fetch recompiled packages in a simplified way.

And too is has much more interesting C++/CMAKE project_options ready to use!