Depending on Package Config file as External Project

Code
1

I’m looking to provide sample code to a library built using CMake. The sample is hosted alongside the sources of the library in a folder called <project_root>/example and I would like it to show how downstreams would use the library that are not part of the build. It doubles as test code for deployment tests in CI. The example has its own CMakeLists.txt which is authored as a top-level CMake sciprt with project() and everything, something like this:

cmake_minimum_required(VERSION
  3.8 # CXX_STANDARD 17
)

project(example1
  LANGUAGES CXX
)

find_package(hpc
  REQUIRED
)

add_executable(${PROJECT_NAME}
  example1.cpp
)

target_link_libraries(
  PRIVATE
    hpc::hpc
)

If I simply traverse this folder from the parent directory such as:

add_subdirectory(library)

if(BUILD_EXAMPLE)
  add_subdirectory(example)
endif()

the find_package() command is going to fail, even if the library already exists. I tried adding it to the build as an External Project like this:

if(${CMAKE_VERSION} VERSION_LESS "3.12.0")
  string(REPLACE ";" "|" CMAKE_PREFIX_PATH_ALT_SEP "${CMAKE_PREFIX_PATH}")
else()
  list(JOIN CMAKE_PREFIX_PATH "|" CMAKE_PREFIX_PATH_ALT_SEP)
endif()

ExternalProject_Add(example1
  PREFIX ${PROJECT_BINARY_DIR}
  SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/example1
  LIST_SEPARATOR |
  CMAKE_ARGS
    "-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}"
    "-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}"
    "-DCMAKE_PREFIX_PATH=${PROJECT_BINARY_DIR}/library/CMakeFiles/Export|${CMAKE_PREFIX_PATH_ALT_SEP}"
  TEST_EXCLUDE_FROM_MAIN ON
  DEPENDS hpc
)

however the generated package config files refer to relative library locations which are invalid inside the build tree.

[build]   The imported target "hpc::hpc" references the file
[build] 
[build]      "C:/Users/mnagy/Source/Repos/hpc/.vscode/build/library/CMakeFiles/Export/lib/hpc.lib"

Indeed, hpc.lib is located in ${PROJECT_BINARY_DIR}/library/hpc.lib

TL;DR

What is the best way to incorporate top-level CMakeLists.txt files into a build that rely on Package Config files generated by the currently running build?

2

I have solved this Problem here: https://github.com/ClausKlein/modern-cmake-sample/tree/feature/test_use_package

2 Likes
3

Thank you Claus, the blog post was a good read too. The design posted there solves a slightly different problem. I’m not looking to separate the build into the CMake and CTest. (The proposed solution does that: it separates building of the example into a unit test. That is how it breaks free from the circular dependency of all depending on example1, example1 depending on install and install depending on all.

The title is slightly misleading, I put myself in a corner. Ultimately I’m trying to invoke a top-level script as part of a build. There are two problems to solve:

  • find_package not crashing and burning
  • hpc::hpc resolving to something meaningful

If I make this script part of the build via add_subdirectory() (multiple project() invocations will still fly) I can lie and define an ALIAS library such as add_library(hpc::hpc ALIAS hpc) and have it resolve to the target in the running build. To make find_package() behave as it should, I’ve handcrafted a fake hpcConfig.cmake in the examples folder which I list(APPEND CMAKE_PREFIX_PATH ${CMAKE_CURRENT_LIST_DIR}) which holds only a single line of defining the library alias.

Conducting deployment tests will be done using CI and not through CTest.

4

Note: find_package can only find an alrady installed package (exported cmake config files)

If you want to use your target hpc in the same project, use

if(NOT TARGET hpc)
    find_package(hpc REQUIRED)
endif()
5

That is also a simple solution, but I did want to provide the sample code ready to be copy-pasted from the library sources: both the .cpp and the build script. Usually one does not guard find_package() commands with the target already existing or not. That’s not something I’d like to promote for my users (or anyone for that matter). If I’m setting an example on how to consume my library as if it were a standalone application, I’d like to keep that part clean.

Adding a one-liner “fake” package config file that isn’t installed, but aliases the library as part of the build tree is sufficiently simple and keeps sample code clean.

6

This is not a simple solution!
It is very important to write cmake code like this.

If you project is used as an ExternalProject, it builds the target without install

see https://github.com/ClausKlein/spdlog/blob/develop/CMakeLists.txt#L153

and

1 Like
7

That’s not actually true. Or, rather, it is true, but only because it’s considered good practice for find and config modules to wrap their target creations in some form of protection against multiple invocations. Usually that takes one of two forms…

# Before attempting to creating any targets
if(TARGET ns::target)
  return()
endif()
# or, before each target is created...
if(NOT TARGET ns::target)
  add_library(ns::target IMPORTED ...)
  ...
endif()

…otherwise, two find_package() invocations that create the same target would crash and burn. Your library’s EXPORTED CMake configs no doubt do this. (If not, they must.)

Here’s what I don’t understand, though: You’re insistent that this example project must both (a) be exactly identical to how your library would be used in dependent code, but also (b) be part of the library build itself.

…But why, on that second constraint? I should be obvious that building a component of a library package is different from building a separate project that depends on that library package. And by “working around” the fact that your library can’t be picked up with find_package() while it’s still being built, you’re also not actually testing (or demonstrating) important aspects of your package, like the EXPORTED CMake configs.

Using CTest to put the library through its paces after installing it to a temporary directory, as Claus has enhanced Pablo’s Modern CMake sample to do, is a clever way to fully address that issue.

Doing the same thing using an external CI is also a perfectly good approach.

But trying to forcing your project to effectively consume itself, Ouroboros-style, just feels counterproductive.

In terms of your library’s potential users, wouldn’t there be more utility in pointing them at the example project after your package is built (and possibly installed)? You can display a message at the end of the build, and/or include instructions in a README:

The libwhatever source includes an examples/client/ directory. This contains the complete source code and CMake project files for libwhatever-client, which uses libwhatever to do whatever a libwhatever does. You can use it to test your libwhatever installation by building the project using CMake:

cmake -B build_example -S examples/client
cmake --build build_example
build_example/libwhatever-client

That way, they’d know that they’ve successfully installed your library and it’s working as expected on their system. That’s something your in-build build can’t possibly show them.

1 Like
8

If people weren’t be doing that, CMake scripts would be noisy as hell. Imagine if in C/C++ land everyone would have to write include guards at the point of inclusion and not inside the headers themselves. It is only natural that find_package() commands may be called multiple times (moreover subsequent invocations are silent).

The second constraint is because I quite passionately dislike how SDK / library sample codes don’t help in getting started with projects. One either starts tinkering with modifying examples as part of the SDK / lib builds or as step 0 rewrite the build scripts from scratch, as example code build definition is part of a multi-layered, much larger build definition. This practice results in users still clinging to the “old ways” and anti-patterns such as:

set(CMAKE_CXX_FLAGS "-lmylib -L/home/user/build/lib")

and other crimes against humanity. I want to set an example on how to properly depend on the installed library. You are right that it may just be a totally separate build, but I did want to cater to dominant practice of expecting sample code to (seem to) be part of the library build. ExternalProject seemed like a good way to make that happen. In the long run I may turn out to regret that (if it won’t scale, due to every example doing a configuration of it’s own), but for <10 examples this should be two birds with one stone.

9

If I’m the one stamping out the ExternalProject invocation (because every author of a downstream is in control of how that invocation looks like), if a project is meant to be consumed it being installed and through the Package Config (as is the desired world order), what prevents me from building the install target of the external project? I specify whatever CMAKE_INSTALL_PREFIX I desire for the external projects, possibly the same as my projects own.

10

FWIW I’m strongly against fetching dependencies using External Project. I consider this to be an anti-pattern, CMake providing some poor man’s dependency management which more often than not backfires. Projects that are keen on pulling in dependencies as external projects almost always don’t guard against a simple find_package() of said library succeeding (because someone actually self-hosts or has that library already installed in default places). If they guard against it, it’s always a different option (no canonical names like BUILD_TESTING for pulling in deps, and is usually an all or nothing switch for better or worse.

IMHO ExternalProject is only a valid invocation when it really is the same project but hosted in different places, eg. like LLVM before moving to the monorepo it is now. When a handful of libraries are really one big project, but for some reason are seperated into multiple build trees and/or repos. CMake build defintions venturing into dependency management is just plain wrong. There are dedicated projects to make that happen (Vcpkg, Conan, Hunter, etc.) and build definitions shouldn’t try to be smart, cause ultimately they do more damage than harm.

The example codes being projects on their own right but still being part of the build is (again, IMHO) one of the rare cases in which ExternalProject is valid.

1 Like
11

You can provide an option to install an external project or not.

You mean projects require dependencies which should be optional? This is an issue with the project not with CMake itself. I never used Vcpkg, Conan, Hunter, etc, what are your requirements for dependency management?

I think you are right, it would be a lot of work to implement a package manager with CMake, for example to find which packages were automatically installed and are no longer required.

12

Yes, that is true. A good point (i.e.) to start with conan and cmake is: https://docs.conan.io/en/latest/getting_started.html

… and it works :wink:

1 Like