Best practice(s) for "isolating" dependencies pulled with FetchContent

Code
1

This in-part has been discussed before, though in a very scattered manner, so I wanted to seek clarification of what I’ve come to find here, all in one place.

In this situation there is a lib, and several applications that depend on this lib. Each project is handled separately and are not inherently part of the same source tree. I control the lib and applications.

My investigation started because in my applications I always want the lib to be built from source, while also being able to use its targets, hence I’m using FetchContent; however, I was surprised to find that when running cmake --install the targets of my lib were also installed along with the app’s targets. After looking into this, I now understand why this happens (tl;dr because FetchContent uses add_subdirectory), so then I started looking into how to best circumvent this behavior.

To be frank, ideally there would be something that was a mix of FetchContent and ExternalProject, where I can still ultimately still include the lib at configure time using find_package(), but instruct CMake how to download, build, and install that package in an isolated context before hand transparently (so that the targets are imported in the exact same way as if I had a local copy of the lib and used find_package). I know something like this is possible by using ExternalProject and a super build, but this requires restructuring and is obviously not at all transparent.

Overall, this led me down the path of how to use my lib with FetchContent, such that I can get as close to the above ideal as possible.

Relevant resources I perused:

I already use the practice of “namespaced” targets to avoid name collisions and ensure a consistent
target interface between consuming the library via find_package() and add_subdirectory():

add_library(project_friendlylibname)
add_library(Project::FriendlyLibName ALIAS project_friendlylibname)
set_target_properties(project_friendlylibname PRROPERTIES
    EXPORT_NAME FriendlyLibName
    ...
)
install(TARGETS project_friendlylibname
    EXPORT FriendlyLibNameTargets
    COMPONENT FriendlyLibName
    ...
)
install(EXPORT FriendlyLibNameTargets
    NAMESPACE Project:::
    ...
)
export(EXPORT FriendlyLibNameTargets
    NAMESPACE Project:::
    ...
)
# Other stripped out stuff...

As for the install issue, without significant restructuring (i.e. super build), it seems like I have two reasonable options:

  1. When consuming, instead of FetchContent_MakeAvailable, do:
if(NOT lib_POPULATED)
    FetchContent_Populate(lib)
    add_subdirectory(${lib_SOURCE_DIR} ${lib_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()

Annoying, since it requires now less clean use of FetchContent in all dependent projects, but if this does the trick then its not really that bad.

P.S. It seems there was a similar solution of using something like set_directory_properties(PROPERTIES EXCLUDE_FROM_ALL YES) before fetching, but this relied on what was ultimately decided as unintended behavior of that directory property, which was removed with !3863.

  1. Within the library I could check for PROJECT_IS_TOP_LEVEL, and if false, add EXCLUDE_FROM_ALL to all relevant invocations of the install command, so that when the all target is installed in the consuming application, none of the library’s components will be installed unless they were explicitly specified (i.e. generally never). These seems reasonable to me, though it may have side effects I’m not thinking of.

So basically, are there any better/other approaches one could recommend for resolving this install issue, and are there any other best practices I should know of for making my library as consumer friendly with FetchContent as possible?

Hoping in the long run we keep getting closer to really straight forward inclusion of dependencies via CMake that other languages have intrinsically.

Can link to lib in question if desired (again, new user link limit).

2

To be frank, ideally there would be something that was a mix of FetchContent and ExternalProject, where I can still ultimately still include the lib at configure time using find_package(), but instruct CMake how to download, build, and install that package in an isolated context before hand transparently (so that the targets are imported in the exact same way as if I had a local copy of the lib and used find_package).

Have you looked at vcpkg? This is exactly why I like it. https://vcpkg.io

3

Hmm, so vcpkg is something that I’ve used once in the past, and I do know that its slowly been on the rise in terms of usability and popularity. I’m not completely sure that it’s a good fit for my use case, but I could very well be wrong.

From what I can remember and quickly gather, it seems like it follows a “packages installed to the system” model, which I mean, is basically what every package manager is. What I like about the FetchContent approach is that the use of the library is contained to the project only, and doesn’t really touch the system.

For large, very popular, or commonly used libs/tools it makes absolute sense to have them installed on my machine (i.e. Qt, Doxygen, git, etc.), but this library is small and only built statically, with more of “it’s just convenient reusable code” kind of feel. Because of this, I’ve enjoyed the non-system install approach, and while I know that FetchContent doesn’t even run install, anything I’d use that does I’d prefer to just install to a folder that’s part of the parent projects build. This is additionally preferred given that I use this lib in many projects, which often use different versions of the lib, and commonly during development use experimental builds via a specific Git commit hash that can be “rapidly” changed.

A lot of example/discussions of vcpkg out there discuss the traditional package manager use of the program:

  • Download and build a package that was already created by the software creator or a maintainer
  • Install the build to local system
  • Make the install available to other things like Visual Studio, CMake, etc.

This usage is great, and I probably would also use it in this “standard” capacity, but for this specific lib not as much. There are bits and pieces I’ve read here and there that give me the impression that vcpkg can be used in the more ad-hoc manner I described above, but if/how isn’t entirely clear to me, along with if/how I can use the tool to obtain my own library off GitHub without having it “professionally” packaged and added to the main vcpkg repo (I think I might be looking for registries? But the MS blog post on the release of those is very focused on closed source software as an example use case so I’m not sure if there’s a different solution for software that is open-source, but not on the main list).

I’m sure some of this I could figure out over time via some research, but since my initial dip indicates that things are a bit uncertain, I’m not totally committed to trying it out yet without being more convinced it’s a good idea. In part this is because I very recently just got up to speed learning CMake, Doxygen, and GitHub Actions and considering the adoption of another major piece of software into my toolchain after having just grounded myself in those makes me a little apprehensive, lol.

If you have any knowledge or references of using vcpkg in such a capacity then I certainly would vale any pointers, but otherwise it’s something I’ll have to poke at here and there and see if I warm up to it as I go along. I definitely appreciate the suggestion though, as it does seem like a real consideration and something worth learning in the long run anyway.

4

If you write a vcpkg.json file with your dependencies, it will run in “manifest mode” which installs packages to your build directory, not vcpkg-wide. Vcpkg never installs anything to your system, except when you ask it to install its Visual Studio integration hooks.

1 Like
5

Huh, I must have been using it in classic mode or something. I’ll look into it now that I know what I should expect at a high level.

Thanks.

6

You can circumvent the installation by setting the default install component name before including the sub project. Together with component based installation, this will simply not install that component if not wanted.

7

Wouldn’t this require that the install directives of my library have no COMPONENTs set (they do)?

Together with component based installation, this will simply not install that component if not wanted.

And by this are you suggesting that when I install the consuming application I use “–components” to explicitly name the application projects target to avoid including the library’s?

If this is the case, setting a default component name wouldn’t even be necessary as the moment I’m not using --install all the ‘Unspecified’ components won’t be included anyway.

Regardless, I’m only really interested in a solution that works transparently from the consumer’s side when compiling (i.e. script changes are ok), such that they can simply build and install their all target like normal (assuming that’s what they do) once their script is written and be done with it. Ultimately it would be preferable if it could all be done on the library side though.

Perhaps I’m misunderstanding what you mean?

For the time being, I’m currently trying out:

and it seems to work quite well, but as quoted there could be consequences in some situation that I’m not aware of.

8

If they already use components, you can choose to not include them. The wanted set can be done in CMake code, no need for the user to do that.

At least that’s how I do it.

Of course, getting component namespaces would be preferred. Same for target namespaces.

9

I think I should clarify exactly what I’m shooting for here as the overall idea is to not the install components become opt-in not opt-out when used as a sub-project.

I’d like it to be that when building the library as the top level project, all components are considered part of the all target, so that

cmake --install .

Will install the entire library (and of course if desired --components can be used to install specific components if desired).

But then if the library is a sub-project in a build tree, e.g. because of FetchContent, I want

cmake --install .

To only install the artifacts of the consuming application, and not install any of the library’s targets, unless explicitly opted into using --components lib_component, since in that context the focal point is the application, not the static lib. It would make more sense to have the targets install if it was a shared lib, and if I ever make a shared version of this library I’ll keep that in mind.

At least for me this is a “sane default”, I wouldn’t want the application builder to have to

but rather choose to include them if desired.

Like I said before the approach I said I’m trying before does seem to do just this. So far so good.

Just wanted to clarify since I’m not 100% what you mean in the context of what I was trying to achieve.

This I’m curious about.

As shown in the OP I do already use namespaces for my targets, including the faux namespaces using the “alias trick” for direct target exports; however, install components don’t support namespaces directly, so are you suggesting I do the same trick for them and set their names to something like `ProjectName::InstallComponent’ so it’s as if they’re behind a namespace?

Hopefully this work makes all of the namespace alignment for each and every part of the project unnecessary eventually: https://gitlab.kitware.com/cmake/cmake/-/issues/22687