ExternalProject, superbuilds and Visual Studio

I could use some help with properly setting up my project in CMake. The main platform is Visual Studio on Windows (but it would be nice to have something portable that works on other systems, esp. Linux + Ninja + gcc). What I have right now is an existing project, let’s call it A, that comprises two shared libraries and an executable. It uses CMake as its build system – it works like a charm, no issues there.

Now I want to set up another project, named B, which uses A (or, to be more specific, shared libraries created by building and installing A). Both A and B have some external dependencies, but all of that is handled by find_package, so it isn’t a problem. What is the problem, is the fact that B uses a different (older) version of VS compared to A. This is because some 3rd party dependencies force the use of an older version of Visual Studio. At the same time, A uses C++17/C++20 internally, so that pretty much requires the latest and greatest versions of MSVC/clang. A has been designed to be usable from older platforms, so the mismatch isn’t really an issue.

I would like to set up B in such way that the generated VS solution can reference A and rebuild (and reinstall) it, if necessary (i.e. A source has been modified). Because the toolchains are different, I don’t think I can use FetchContent here, despite A being CMake-based – which realistically leaves the ExternalProject route (assuming I want to stay in the CMake land).

I’ve run some basic experiments on using ExternalProject_Add to handle A as a dependency in B and it looks promising so far. But here is where I’ve run into some issues. I’ve been reading Craig Scott’s book on CMake and it describes superbuilds as one of the recommended ways of handling this situation: I did some googling and found this example in Chromium codebase: https://chromium.googlesource.com/external/github.com/grpc/grpc/+/HEAD/examples/cpp/helloworld/cmake_externalproject/CMakeLists.txt

What it amounts to, is basically writing a script that pulls in both A and B via ExternalProject_Add. B then could be simply written to find_package A. While this works fine for our automated builds, the developer experience leaves something to be desired, as the generated solution seems to only contain references to A and B with some CMake rules on how to configure/build/install each project, while I would prefer to have B set up more like a traditional VS project (with includes and sources in the project, neatly grouped using filters) and keep A as an external reference. The only way I could achieve something like that was by using ExternalProject_Add to add A directly in the CMake script for B, but that I think would force me to mix the targets defined by ExternalProject and the ones I define in B, which seems to be strongly discouraged by Craig.

Is there some clever solution I’ve missed? Is there a way to use the superbuild script AND have the typical VS project for B in the same solution working nicely? Should I just forget about superbuild in this case?

1 Like

Yes, there is a piece of the puzzle one could say you missed: ExternalProject_Add creates a standalone VS solution for the project being added. The normal course of action when using a superbuild (the recommended usage of ExternalProject) is as follows:

  1. Generate the superbuild project.
  2. Open the .sln of the superbuild project and build it. This will build and install A, and also generate (and build) project B.
  3. Close that .sln, and instead open the .sln generated for B. Do further development using that. Your dependencies are already correctly set up and installed by the build of the superbuild project in the previous step.

So while there is no way to have “configure A” and “develop B normally” in the same .sln, you can “configure A” and “develop B normally, with dependecies to A correctly set up.” This is (AFAIK) the intended use for ExternalProject superbuilds, and from my own experience works fine.

1 Like

If there’s no chance of wanting to expand things further with more projects than just A and B, then you could get away with this. One of the reasons I discourage defining B in your top level project while pulling in A as an ExternalProject is that when some other project C later wants to pull in B, it doesn’t have control over how A is brought into the build. If C pulls in some other project D which also pulls in A, then it gets difficult to make it all work together (B and D both separately pull in A).

A traditional superbuild arrangement keeps each project standalone, expecting each of its dependencies to be supplied via a separate build. The top level of the super project then just does coordination and nothing else. The reply from @Angew is good advice for how to work with B in a traditional super build arrangement.

Thanks!

Ended up with a mixed approach: an option of pulling A into B via ExternalProject_Add, as it results in a solution that is slightly more convenient to use during development, and a proper superbuild arrangement for our CI/CD builds (as it’s very likely we’ll have more projects that depend on A, so a superbuild, being way more clean, is definitely the way to go).

BTW. Absolutely love the book, it’s incredibly well written!

1 Like