Since CMake 3.24 it seems possible to implement the following feature within the CMake framework:
First, look for a local pre-compiled library in the system or a specific position using CMake find_package tool.
If it is unavailable, download the library from a specific position, e.g. github or some other repositories, build it, and provide the target.
This looks fantastic, yet when I am trying to import some non-CMake libraries, e.g. jemalloc, I notice that the FetchContent will properly download the code in the config part and ignore the CONFIGURE_COMMAND, BUILD_COMMAND and INSTALL_COMMAND. They are also expected. Now here is the hard part:
find_package will not find jemalloc in a clean docker environment.
Then, FetchContent will be triggered and download the code
Yet, FetchContent_MakeAvailable will only populate the directories, not targets.
I still need something, maybe ExternalProject or execute_process, to build the code.
It seems vcpkg, conan or other package managers cannot be introduced to our CMake system for reasons I do not know.
Our situation looks like this:
Internally, we build the code in a Docker environment for various reasons. Inside the environment, every library must be pre-built. CMake should use find_package to locate the corresponding targets.
For people outside our company, in convenience CMake should download the libraries if they are not found or not the correct version. The libraries include jemalloc, boost, and others that are not CMake-ready.
I am wondering about the right approach for this situation; maybe FetchContent first and then ExternalProject_Add? I have other thoughts, e.g. use add_custom_command after FetchContent_MakeAvailable.
No, the CONFIGURE_COMMAND, BUILD_COMMAND, etc. are explicitly documented as not supported by FetchContent. It discards any such options and forces them to be empty strings. An explicit goal of FetchContent is to avoid building at configure time. You’re incorrectly expecting FetchContent to have all the capabilities of ExternalProject, but they have different goals and are aimed at different use cases. FetchContent can only add CMake-based projects directly to the main project. If you use it with a non-CMake dependency, all FetchContent can do for you in that case is download the sources, but not add it to your main project’s build, which is what you’ve observed. A non-CMake dependency can really only be added using ExternalProject.
Based on your scenario, it sounds like your internal Docker environment builds are fine, but builds by people outside your company are where you’re hitting problems. For those, I’d advise don’t try to do too much and leave those users to decide which approach for providing the dependencies works best for them. Just have the find_package() calls in your project. Let the end user decide how those should be satisfied. They might choose to use a package manager, or they might pre-build them and make them available by modifying CMAKE_PREFIX_PATH. Or they might implement their own wrapper to build them with scripts. Or… whatever other esoteric approach works for them. For dependencies that do build with CMake, there’s nothing wrong with your project calling FetchContent_Declare() with the FIND_PACKAGE_ARGS keyword and calling FetchContent_MakeAvailable(). That still leaves all options open for your non-internal users to provide the dependency using their preferred method. For the non-CMake dependencies though, I don’t think FetchContent is what you should be using in this situation.
Thanks for your suggestion! Leaving the external library issue to other people might be a good solution. Yet my problem is that we do have many libraries already configured using ExternalProject_add and I will face a lot of complaints if those libraries are not “automatically” downloaded. Do you think in this case it is possible to combine find_package with ExternalProject_add? If not, may I know what might be the alternative approach?
I did think about this, but the problem lies here:
CMake community encourages jemallocConfig.cmake.in and not accepting new Findjemalloc.cmake into its standard modules. I interpret this as “we should avoid Findxxxx” as much as possible.
To some extent, I feel the Finder should not be a “builder”.
Besides the issues above,
To my understanding, Findxxx works on the configuration time, implying the external project has to be built during that time. I suppose this might not be favored.
The main problem you will have with trying to combine find_package() and ExternalProject is that find_package() provides details at configure time, but ExternalProject only provides them at build time. You’ll end up having to manually define all the targets and the underlying files they correspond to, along with all compiler flags, header search paths, etc. Basically all the usual disadvantages of ExternalProject when used in anything other than a pure superbuild arrangement.
Wouldn’t this be on the project level rather than to be built into the standards? For cmake projects we should strive for config ones, but some projects are dead set on not supporting cmake. I don’t see a good way to incorporate with those other than FindPackage.cmake. For non-cmake projects I think it makes sense to combine them there.
Also because FindPackage.cmake is generally unregulated, I think it is on equal footing with FetchContent and ExternalProject wince for those you also should define the compilation options.