FetchContent a directory but add a subdirectory

I have some code using git submodules that is:

  1. Working on a full checkout of the highs library, let’s say in deps/highs.
  2. Calling add_subdirectory(deps/highs/src).

The deps/highs/src/CMakeLists.txt file contains references to ${HIGHS_SOURCE_DIR}, which is deps/highs, i.e. the parent directory.

Is there a way to accomplish the same functionality with FetchContent? I.e. download everything but add only src subdirectory. Because using SOURCE_SUBDIR src in FetchContent_Declare will change ${HIGHS_SOURCE_DIR} to build/_deps/highs/src, won’t it? I have tried that and it doesn’t seem to do the work.

Thanks!

Can you just set it via -D? Skipping the top-level logic doesn’t seem like it’s something the project really supports.

1 Like

Hi @ben.boeckel,

I’m sorry but I don’t understand your answer. Could you please elaborate a bit more on it?

Thanks!

With FetchContent, add -DHIGHS_SOURCE_DIR=<SOURCE_DIR> to the CMAKE_ARGS.

1 Like

OK, thanks!

I could hardcode that, yes, but I would also need to hardcode HIGHS_BINARY_DIR, and I wouldn’t be sure yet I would be missing something else.

Yeah, that’s a likely one. I’d take a look at the CMakeLists.txt to see what logic you might be skipping over at least.

1 Like

Sorry again, @ben.boeckel , but I don’t see how setting -DHIGHS_SOURCE_DIR can help here.

  • FetchContent will set HIGHS_SOURCE_DIR to the root folder.
  • If I tell CMake that HIGHS_SOURCE_DIR is the src folder within root, then, when it configures Highs, and it gets to src/CMakeLists.txt, the code in there will expect HIGHS_SOURCE_DIR to point to the root folder.

Couldn’t I use just FetchContent_Declare (or ExteranalProject_Add) plus add_subdirectory("${HIGHS_SOURCE_DIR}/src")?

Why would it do that? These variables usually come from a project(HIGHS) call.

1 Like

Oh, sure, I wasn’t seeing that. FetchContent will set highs_SOURCE_DIR but not HIGHS_SOURCE_DIR. You’re right, thanks!

So is your suggestion then:

  • FetchContent with SOURCE_SUBDIR pointing to src, plus
  • Setting HIGHS_SOURCE_DIR to `${highs_SOURCE_DIR}/…" and “HIGHS_BINARY_DIR” to “${highs_BINARY_DIR}/…”?

It seems the code below is working.

It uses your suggestion of setting HIGHS_SOURCE_DIR (BTW, the project I am modifying, which was managing dependencies via git submodules, was already doing that).

But it also uses:

  • FetchContent_Declare, to declare dependencies,
  • FetchContent_Populate, to download dependencies and set highs_SOURCE_DIR and higs_BINARY_DIR,
  • set(HIGHS_SOURCE_DIR) and set(HIGHS_BINARY_DIR), the two variables that are accessed by src/CMakeLists.txt and that have to point to the parent dir of src, and
  • add_subdirectory(src), to process only src subdir.
FetchContent_Declare(highs
    GIT_REPOSITORY https://github.com/ERGO-Code/HiGHS.git
    GIT_TAG "45a127b78060942721f75f46a04b274c2bb963d8"
)
FetchContent_Populate(highs)
set(HIGHS_SOURCE_DIR "${highs_SOURCE_DIR}")
set(HIGHS_BINARY_DIR "${highs_BINARY_DIR}")
add_subdirectory("${highs_SOURCE_DIR}/src" "${highs_BINARY_DIR}/src")

I might try doing the same with:

FetchContent_Declare(highs
    GIT_REPOSITORY https://github.com/ERGO-Code/HiGHS.git
    GIT_TAG "45a127b78060942721f75f46a04b274c2bb963d8"
    SOURCE_SUBDIR src
)
set(HIGHS_SOURCE_DIR "${highs_SOURCE_DIR}")
set(HIGHS_BINARY_DIR "${highs_BINARY_DIR}")
FetchContent_MakeAvailable(highs)

The first of the solutions from my last post, the one using add_subdirectory, has the problem that the subfolder is treated like a project’s folder. So, for example, if I build my project with warnings as errors, and the subproject raises some warnings, those will be treated as errors of my project.

Fortunately, it can be avoided using CMake 3.25’s add_subdirectory(... SYSTEM).

Avoid calling FetchContent_Populate() directly. I plan to eventually deprecate that. Call FetchContent_MakeAvailable() instead.

Note also that you can add the SYSTEM keyword to FetchContent_Declare() starting with CMake 3.25 as well.

1 Like

That’s the wrong order. The highs_SOURCE_DIR and highs_BINARY_DIR variables won’t be populated until after the call to FetchContent_MakeAvailable().

EDIT: Oh, I see that reordering would be a problem in this case. You need HIGHS_SOURCE_DIR and HIGHS_BINARY_DIR to be populated before the downloaded source has been loaded. You could let FetchContent handle the downloading only, but take control of the add_subdirectory() part by setting SOURCE_SUBDIR to a non-existent subdirectory. That will prevent FetchContent_MakeAvailable() from calling add_subdirectory(). The main advantage to using FetchContent_MakeAvailable() is for the dependency provider and find_package() integrations, which might not be important to you right now, but may be in the future or for other consumers of your project.

1 Like

Many thanks for your comments @craig.scott.

Are you then proposing something like this?

FetchContent_Declare(highs
    GIT_REPOSITORY https://github.com/ERGO-Code/HiGHS.git
    GIT_TAG "45a127b78060942721f75f46a04b274c2bb963d8"
    SOURCE_SUBDIR this-directory-does-not-exist
)

# FechContentMakeAvailable will set
# highs_SOURCE_DIR to _deps/highs-src/this-directory-does-not-exist, and
# highs_BINARY_DIR to _deps/highs-build/this-directory-does-not-exist
FetchContent_MakeAvailable(highs)

# Properly setting
# highs_SOURCE_DIR to _deps/highs-src/src, and
# higs_BINARY_DIR to _deps/highs-build/src
set(highs_SOURCE_DIR "${highs_SOURCE_DIR}/../src")
set(highs_BINARY_DIR "${highs_BINARY_DIR}/../src")

# Now carry on doing what we were doing with FetchContent_Populate
set(HIGHS_SOURCE_DIR "${highs_SOURCE_DIR}/..")
set(HIGHS_BINARY_DIR "${highs_BINARY_DIR}/..")
add_subdirectory("${highs_SOURCE_DIR}/src" "${highs_BINARY_DIR}/src" SYSTEM)

OK, I had it working with the following code:

FetchContent_Declare(highs
    GIT_REPOSITORY https://github.com/ERGO-Code/HiGHS.git
    GIT_TAG "45a127b78060942721f75f46a04b274c2bb963d8"
    SOURCE_SUBDIR this-directory-does-not-exist
)
FetchContent_MakeAvailable(highs)
set(HIGHS_SOURCE_DIR "${highs_SOURCE_DIR}")
set(HIGHS_BINARY_DIR "${highs_BINARY_DIR}")
add_subdirectory("${highs_SOURCE_DIR}/src" "${highs_BINARY_DIR}/src" SYSTEM)