FetchContent in dependency management

I would like to ask for some guidance on best practices and suggestions on how to handle deps in CMake. I am developing CMake-based C++ projects for customers that depend on a couple of libraries but wish to provide a clean out-of-the-box experience for the consumers of these projects. This means, that git clone, configure, build should succeed without end users lifting a finger.

The solution in one case was using Craig Scott’s DownloadProject, which has a crucial shortcoming, which FetchContent is supposed to address (and it does, bur more on that later). The solution in another case using git submodules is pulling in the source code of dependencies even before configuration would start. I also tried using the ExternalProject module in cases, but having to deal with setting up INTERFACE targets for every dep manually is tedious, error prone and not always sufficient, due to builds still missing some files at configure time, sometimes needed as sources, sometimes to decide link language, etc.

My experience is none of these solutions scale. DownloadProject, ExternalProject, FetchContent, git submodules… they all turn into intricate fixups of other project’s mess.

IMHO all of these methods should only be used when the dependency is really part of the same project.

Git submodules are for projects that have multiple… modules, and haven’t transitioned to a monorepo. Same goes for FetchContent and similar CMake modules. They only work reliably when one is in total control of everything pulled in. In all other cases, a singular call to find_package(MyDep REQUIRED) is the only level of concern downstreams should have to deal with. Much like Find Module and Package Config files, these need not be guarded against the targets already existing, because these files are safe for multiple inclusions. The idiom

if(NOT TARGET glm::glm)
  find_package(glm CONFIG REQUIRED)

is a sad consequence of the new practice of pulling in dependencies into our own build. find_package(glm CONFIG REQUIRED) only conflicts with glm when it’s pulled in as part of the build, because then the ALIAS and INTERFACE targets would clash. What’s really annoying is that every project has to guard it’s own targets too:

  add_library(MyLib ...)

Because no project knows whether they’re pulled into the build multiple times by a superproject. Moreover, when trying to cater to those both supplying their own deps through APT or Vcpkg, etc and also those enjoying our custom crafted out-of-the-box experience, one has to deal with the subtleties of ALIAS vs. INTERFACE targets, as they have different sets of properties. Here is one of the less complicated ones:

  find_package(glm CONFIG)
  # NOTE 1: GLM in Ubuntu 18.04 repo doesn't install the IMPORTED
  #         INTERFACE target, only the legacy variable is defined in glm-config.cmake
  # NOTE 2: auto-fetched subproject build doesn't define the (legacy) variable
  #         anymore, only the INTERFACE target
  # To avoid every test depening on GLM define their deps using
  # add_sample(
  # LIBS
  #   $<$<TARGET_EXISTS:glm::glm>:glm::glm>
  #   $<$<NOT:$<TARGET_EXISTS:glm::glm>>:"${GLM_INCLUDE_DIRS}">
  # )
  # we create the INTERFACE target in case it didn't exist.
  if(glm_FOUND AND NOT TARGET glm::glm)
    add_library(glm::glm INTERFACE)
    target_include_directories(glm::glm INTERFACE "${GLM_INCLUDE_DIRS}")

if(NOT (glm_FOUND OR TARGET glm::glm))
  if(NOT EXISTS "${CMAKE_BINARY_DIR}/_deps/glm-external-src")
      message(STATUS "DEPENDENCIES_FORCE_DOWNLOAD is ON. Fetching glm.")
      message(STATUS "Fetching glm.")
    message(STATUS "Adding glm subproject: ${CMAKE_BINARY_DIR}/_deps/glm-external-src")
  cmake_minimum_required(VERSION 3.11)
    GIT_REPOSITORY      https://github.com/g-truc/glm
    GIT_TAG    # e79109058964d8d18b8a3bb7be7b900be46692ad

25 lines of script (without comment) for one dep. And I got 5-6-7 deps, each with subtly different annoyances:

  • some abandoned where I have to patch the 8 year old CMake scripts,
  • having to scope their build because they don’t build warning-free, so I have to patch up CMAKE_CXX_FLAGS for the deps only to remove warning-related flags for all supported compilers,
  • My favorite is needing to find the location of headers of the target, because they are shader sources as well and I need to copy next to my executable. Finding the path to those for both the ALIAS and IMPORTED cases I still haven’t figured out.
  • Some projects (SFML) don’t support consuming projects as part of the build (freetype), only in pre-built binary mode using find_package,
  • sometimes other shenanigans.

Bottom line is: I feel like this brave new world of using FetchContent for handling dependencies, and having to self-defend all our scripts against superprojects potentially multiply including them, etc. is a dead end. I am aware that 3.24 has some degree of find_pacakge() integration for FetchContent, but I’m not sure that will resolve this duality. This trend of pulling other people’s stuff into our build either at build or either at configure time has to stop.

(I’m also aware of dependency providers, but that requires mandating minimum of 3.24 (not a short term solution) and also getting all of our customers on board of using package managers. I’d absolutely love the idea of that, but I’m afraid that’s easier said than done. At best I can try to detect from script whether any package manager is being used, and if not, FetchContent Vcpkg and in one go install all the deps during configuration time and use everything through the oneliner find_package interface.)

1 Like

Would you consider conan out of scope? With the conan cmake integration, the impact on our library consumers was smaller.

1 Like

Sorry for the radio silence on this one. I intend to respond to this properly, which will take some time. I have to go through this for my Professional CMake book updates, but I’m not up to that part yet. I need to prioritise other 3.24 work first.

The 3.24 release significantly expands the landscape of what’s possible in this area, and I want to avoid giving a flippant answer. I’ll come back to this when I can, but it might be a few weeks.


It could be a solution, or something similar for any of the other package managers. (I’m the maintainer of a couple of Vcpkg ports, so I’d sooner wrap that, just due to familiarity.) This is borderline “use a package manager or I’ll use one for you”.

Thank you Scott for the reply. I’ve been a happy user and evangelist of your book (bought it around the 3.16 timeframe, unsure, didn’t keep a copy of the old files). I’ve read the latest docs of FetchContent and I’ve too seen that significant new functionality dropped in 3.24, but all my projects are either very infrastructure HPC stuff (so ultra conservative and still stick to CMake 3.1, so they can compile on custom Commodore64 rigs) or part of gigantic corporate codebases and I don’t get to call the shots on bumping version reqs, even if for my supplied codebases they mean the world. 3.17 is already sunshine and happiness, but 3.24 is still 18-24 months from hitting cmake_minimum_required.

All in all this stop-gap solution of making things part of our build is a dead end, if you ask me. The 7-8 dependencies I tried all bleed for different reasons and I got the scars to prove it.

  • The OpenCL-SDK is one of them. Dependencies.cmake traverses the Dependencies folder, and there’s the zoo of fixups and hosting patches to build scripts to make things work.
  • The other publicly available is rocPRIM, but I haven’t upstreamed the move to FetchContent yet.
    • Yes, another favorite is GCC’s ParallelSTL depending on Intel TBB, but very specific versions. Detecting CMAKE_CXX_COMPILER_ID isn’t enough, because it’s a dep only when using libstdc++ and not libc++, so one needs to check_symbol_exists and if going bulletproof, check for libstdc++ version to know which TBB version to fetch. GCC just has to ruin it for everybody. Why must I have to jump through flaming hoops for the audacity of depending on the STL? (But of course GCC 9, the first to support the ParallelSTL depends on a TBB version that doesn’t build with CMake, just to make it fun having to build it with custom configure/build/install steps.)
    • GCC is the reason I hate using the ParallelSTL, because I have to explain all this garbage to wannabe physicists using Linux, why they must suffer when building sample code. Even though Canonical provides ppa:ubuntu-toolchain-r/test which makes it easy to install new compilers, one can’t use them without getting into CMake deps, minimally this.

This question is only relevant for v3.24 but…
Do you know if there are any example dependency providers available for vcpkg or conan?
e.g. vcpkg_provide_dependency() , conan_provide_dependency()

Is this subject discussed in your book?
If so, what chapter / section?

I’m not aware of vcpkg or conan having support for the new dependency provider API yet. The API hasn’t been in an official release, so I wouldn’t expect one to be implemented until the 3.24 release has been out for a while (if they choose to support the new API).

I do intend to cover this new feature in the next edition of my book, but until some dependency providers add support for the new API, it will be somewhat limited in what it can demonstrate.

1 Like

First, a bit of general commentary which gives some background for the broader audience: fetchcontent vs vcpkg, conan? - #3 by craig.scott

TLDR: I think this is a case of choose the right tool for the right job. From what I see, you’re looking for the benefits of a package manager, but without having to actually rely on one.

Your desire to have a simple out-of-the-box experience for your customers that doesn’t require anything more than doing a git clone, configure and build doesn’t leave you with many good choices in this case. I think you’re going to have to compromise somewhere, you’re just going to have to decide where you’re willing to accept that compromise.

If a dependency isn’t well-behaved when incorporated directly into a larger parent build, then FetchContent might not be the right fit for that dependency. The sort of measures you show in your original post are not indicative of what I see in projects that use FetchContent. I think you’re in the unfortunate situation where your dependencies just don’t suit that scenario very well. FetchContent shouldn’t be viewed as the tool for every job. It works well for dependencies and scenarios that meet its requirements. Many do, but quite a few don’t. You need to choose the right strategy for the situation.

It sounds like the dependencies you need to support are not very friendly to being added directly as sources. If they expect to be the top level project, built and installed somewhere for other projects to consume as binaries, then a package manager may be the more suitable path for you. That would allow you to essentially outsource patching and maintaining support for those dependencies. I would recommend against requiring a particular package manager if you can, which means don’t hard-code things like package manager setup files or commands. If you stick to find_package() calls, most package managers will typically be able to handle that, assuming they provide the dependencies your project requires. That would also not lock anyone out of making use of the new CMake 3.24 features if they so wished in their own projects that consume yours.

I’d recommend not trying to do too much for the user like this. Such enforced logic has a habit of getting in the way for some users who might do things you hadn’t considered. Trying to robustly detect whether a package manager is used seems like inviting trouble to me too. Rather than forcing such logic on your consumers, I would advise to stick to conventional workflows and provide clear, concise instructions in your project’s README.md for the common package managers your users may be willing to try.

And since you mentioned that many of your customers are from the HPC space, I’d suggest taking a look at Spack, if you haven’t already. It is quite popular with that user community and is actively supported. It may present a path that (a subset of) your users may be happy with.

Not sure that any of the above is telling you things you didn’t already know. Unfortunately, this seems like one of those cases of “pick your poison”.