# Built in package manager for CMake modules
**URL:** https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513
**Category:** Development
**Created:** [February 21, 2023, 8:53pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513 "2023-02-21T20:53:11Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@buildSystemPerson](https://discourse.cmake.org/u/buildSystemPerson)
#### Post date: [February 21, 2023, 8:53pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/1 "2023-02-21T20:53:11Z")
# Problem Statement
Currently there is no standard way to consume external CMake code.
Which makes it challenging to share CMake code. Which is problematic for organizations, companies, package managers, etc.
## Example 1: vcpkg
Lets take vcpkg as an example.
Currently the recommended strategy for consuming vcpkg is to make it a submodule for your project. At one point FetchContent was recommended. But that recommendation was removed:
- [[docs] Revert FetchContent section by ras0219-msft · Pull Request #27311 · microsoft/vcpkg · GitHub](https://github.com/microsoft/vcpkg/pull/27311)
As I understand the FetchContent recommendation was removed for performance, and FetchContent not working well if used before the first project call ([FetchContent / CMAKE\_MAKE\_PROGRAM issue - #6 by ben.boeckel](https://discourse.cmake.org/t/fetchcontent-cmake-make-program-issue/2522/6)).
As a result the majority of users consume vcpkg with a submodule. Which isn’t an ideal workflow for everyone.
## Example 2: cpm (CMake package manager)
CPM unlike vcpkg only has 1 file with all the logic it needs. This is likely to make it easier to distribute to users.
CPM recommends multiple methods for using it. Either copy pasting the file into your project. Using wget. Or writing some CMake code to grab it for you.
- [Downloading CPM.cmake in CMake · cpm-cmake/CPM.cmake Wiki · GitHub](https://github.com/cpm-cmake/CPM.cmake/wiki/Downloading-CPM.cmake-in-CMake)
```auto
set(CPM_DOWNLOAD_VERSION 0.34.0)
if(CPM_SOURCE_CACHE)
set(CPM_DOWNLOAD_LOCATION "${CPM_SOURCE_CACHE}/cpm/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
elseif(DEFINED ENV{CPM_SOURCE_CACHE})
set(CPM_DOWNLOAD_LOCATION "$ENV{CPM_SOURCE_CACHE}/cpm/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
else()
set(CPM_DOWNLOAD_LOCATION "${CMAKE_BINARY_DIR}/cmake/CPM_${CPM_DOWNLOAD_VERSION}.cmake")
endif()
if(NOT (EXISTS ${CPM_DOWNLOAD_LOCATION}))
message(STATUS "Downloading CPM.cmake to ${CPM_DOWNLOAD_LOCATION}")
file(DOWNLOAD
https://github.com/TheLartians/CPM.cmake/releases/download/v${CPM_DOWNLOAD_VERSION}/CPM.cmake
${CPM_DOWNLOAD_LOCATION}
)
endif()
include(${CPM_DOWNLOAD_LOCATION})
```
While this can work it’s not ideal for both users of CPM or the author of CPM.
# Solution Proposal
A built in CMake package manager that borrows the syntax of FetchContent.
However, it does NOT call ExternalProject under the hood. Because we are not trying to build C++ projects. We are just trying to incorporate CMake modules.
## vcpkg
```auto
cmake_minimum_required(VERSION 3.XY)
cmake_module_manager(vcpkg
GIT_REPOSITORY https://github.com/microsoft/vcpkg/
GIT_TAG 2022.09.27
GIT_SHALLOW OFF # vcpkg needs full history
)
set(CMAKE_TOOLCHAIN_FILE "${vcpkg_SOURCE_DIR}/scripts/buildsystems/vcpkg.cmake")
project(FOOBAR LANGUAGES "CXX")
```
## cpm
```auto
cmake_minimum_required(VERSION 3.XY)
project(FOOBAR LANGUAGES "CXX")
cmake_module_manager(CPM
GIT_REPOSITORY https://github.com/cpm-cmake/CPM.cmake
GIT_TAG v0.38.0
MODULE_PATH cmake/ # Append to CMake module path
)
include(CPM)
CPMAddPackage("gh:fmtlib/fmt#7.1.3")
CPMAddPackage("gh:nlohmann/json@3.10.5")
CPMAddPackage("gh:catchorg/Catch2@3.2.1")
```
Ideally there is an easy way to cache/delete what `cmake_module_manager` retrieves for the user. So that CI doesn’t constantly have to fetch the same thing. Which means downloads must be redirectable outside the `BINARY_DIR`.
Taking inspiration from CPM I think a combo cache/environment variable would address this.
- [GitHub - cpm-cmake/CPM.cmake: 📦 CMake's missing package manager. A small CMake script for setup-free, cross-platform, reproducible dependency management.](https://github.com/cpm-cmake/CPM.cmake#cpm_source_cache)
Or perhaps vcpkg’s caching solution is more appropriate:
- [Binary Caching | Microsoft Learn](https://learn.microsoft.com/en-us/vcpkg/users/binarycaching)
Thoughts?
- @TheLartians for CPM
- @Neumann-A for vpckg
---
### Author:  [@Neumann-A](https://discourse.cmake.org/u/Neumann-A)
#### Post date: [February 21, 2023, 9:14pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/2 "2023-02-21T21:14:42Z")
You are still free to use FetchContent with vcpkg. The main issue was that deleting the build folder also deletes the fetched vcpkg checkout. Your approach doesn’t fix that and I also deem it necessary. If you don’t want to use a git submodule just use FetchContent instead. Nobody is stopping you from doing that ([https://cmake.org/cmake/help/latest/module/FetchContent.html#id14](https://cmake.org/cmake/help/latest/module/FetchContent.html#id14)).
Also passing `SOURCE_DIR` might fix the original issue with the FetchContent approach.
---
### Author:  [@buildSystemPerson](https://discourse.cmake.org/u/buildSystemPerson)
#### Post date: [February 21, 2023, 9:18pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/3 "2023-02-21T21:18:46Z")
> [@Neumann-A](#):
>
> The main issue was that deleting the build folder also deletes the fetched vcpkg checkout. Your approach doesn’t fix that
I’m confused what you mean. I explicitly say that `cmake_module_manager` should be able to redirect downloads outside the build folder.
> Ideally there is an easy way to cache/delete what `cmake_module_manager` retrieves for the user. So that CI doesn’t constantly have to fetch the same thing. Which means downloads must be redirectable outside the `BINARY_DIR`.
---
### Author:  [@kisielk](https://discourse.cmake.org/u/kisielk)
#### Post date: [February 21, 2023, 9:28pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/4 "2023-02-21T21:28:22Z")
I like this idea. In our organization we are working on a multi-repo project and we are finding the need to have CMake modules that are used in several repos. Currently that means we are just copying the same `.cmake` files to the `CMake` directories of each project. I think this module manager would be a good way to solve that problem.
---
### Author:  [@Neumann-A](https://discourse.cmake.org/u/Neumann-A)
#### Post date: [February 21, 2023, 10:15pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/5 "2023-02-21T22:15:57Z")
```auto
include(FetchContent)
FetchContent_Populate(
vcpkg
GIT_REPOSITORY https://github.com/microsoft/vcpkg.git
GIT_TAG 703a8113f42e794534df5dfc5cece5dabcb949d0
SOURCE_DIR "${CMAKE_SOURCE_DIR}/vcpkg"
QUIET
)
list(APPEND CMAKE_PROJECT_TOP_LEVEL_INCLUDES "${vcpkg_SOURCE_DIR}/scripts/buildsystems/vcpkg.cmake")
list(APPEND CMAKE_TRY_COMPILE_PLATFORM_VARIABLES CMAKE_PROJECT_TOP_LEVEL_INCLUDES)
```
So I don’t see why a new function is necessary. Same can be done with cmake modules etc.
---
### Author:  [@craig.scott](https://discourse.cmake.org/u/craig.scott)
#### Post date: [February 21, 2023, 10:46pm UTC](https://discourse.cmake.org/t/built-in-package-manager-for-cmake-modules/7513/6 "2023-02-21T22:46:42Z")
The [Dependency Providers](https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html#dependency-providers) feature added in CMake 3.24 is the intended way forward for package managers to integrate with CMake. Package managers would provide a file that users can specify in their `CMAKE_PROJECT_TOP_LEVEL_INCLUDES` variable. This intentionally avoids having to hard-code anything in the project related to any particular package manager to use or not use.
> [@buildSystemPerson](#):
>
> …FetchContent not working well if used before the first project call…
It does work well before the first project call for some use cases. I do this all the time in client projects to bring in a repo that defines common CMake build logic, provides toolchain files, etc. before the first `project()` call. At the moment, it is less suitable if the repo you want to bring in is large because, as identified in the linked issues, a copy is downloaded in each build directory you use. If you blow away the build directory, you will have to download that repo again the next time you run CMake on your project.
Caching of downloaded things has come up before a few times. I’ve resisted this in the past on the basis of not wanting CMake to grow beyond its primary focus. It shouldn’t attempt to become a package manager in its own right. But recently I’ve been reassessing my position on the caching side specifically. I am cautious about how we might provide some sort of caching, seeing the problems that I’ve encountered with this in my own use of package managers and employing certain patterns with client projects. It is clear that having a cache that sits outside the build directory would solve some increasingly common problems users are facing.
> [@Neumann-A](#):
>
> Also passing `SOURCE_DIR` might fix the original issue with the FetchContent approach.
I’ve indeed done exactly that in at least one client project that uses vcpkg. It requires an assumption about where you want to put your vcpkg directory, and you have to think about who/what is going to manage that repo after it is first populated, but it can be made to work if your constraints and assumptions are compatible with that approach. If the caching I mentioned above was implemented, that caching would be a better longer term solution though.
> [@Neumann-A](#):
>
> So I don’t see why a new function is necessary.
I agree, but my view is even stronger than that. As a general comment, we’d like to avoid introducing yet another new command for specifying the packages/dependencies to bring into a project. We already have `find_package()` and `FetchContent_MakeAvailable()`, with the former being the most canonical way to do it. Any new functionality should be able to continue using those. Projects should not need to be updated to take advantage of new ways of providing dependencies. The Dependency Providers feature was developed to carefully follow that principle.
What’s missing at the moment is a way to define details of a dependency in a way that is independent of the implementation used to provide the dependencies. CMake currently provides that capability only for FetchContent via `FetchContent_Declare()`. Vcpkg uses manifests. Conan has its own approach. CPM defines new commands which use FetchContent under the hood. I tried to capture a unified way of specifying dependencies in [issue 22686](https://gitlab.kitware.com/cmake/cmake/-/issues/22686), but I am not optimistic it will gain the necessary support by the major package managers. Private feedback I’ve received expressed concern that it wouldn’t allow implementation-specific details to be specified. I haven’t pursued that discussion any further due to lack of time, but also from having the will to engage with it sucked out of me by the effort it took to get the dependency-related features finally implemented in CMake 3.24.