Out of source builds and caching external assets

Proposal:

ExternalProject and FetchContent modules re-download external archives instead of caching them out of source and out of build. That is sub-optimal for two reasons:

a) I might lose connectivity. Or I might work offline. Or the download server goes offline or dies terminally. Tough luck if I accidentally wipe the build.
b) When working on build issues, regular wipes of the build cannot be avoided, hence creating quite an unnecessary load on external infrastructure like github.

Second, out of source builds could be done by default whenever there is a corresponding env variable set. No more defining the build directory. Those two issues could be combined via one env var that defines the out of source build root dir, ie. $CMAKE_OOS_DIR. Downloaded assets could be stored by default under $CMAKE_OSS_DIR/.cache and builds could go to $CMAKE_OSS_DIR/$PROJECTNAME (or similar).

Thanks.

1 Like

In my ExternalProject builds, I have them directed to store to ${CMAKE_BINARY_DIR}/downloads. On my machine, I symlink this to a common store between a variety of superbuilds. There’s been discussion to have a user-wide cache for ExternalProject sources (which I assume will help FetchContent as well), but it’s not a simple thing (especially for VCS sources).

See this discussion and links from it: ExternalProject_Add cache for git repositories

1 Like

Yes, I don’t care about VCS sources. Let them be how they are for now. This proposal makes mostly sense for archives. And you can download individual revisions from github via archive, too. Supporting caching of archives is better than nothing.

I already wrote a workaround: GitHub - jjYBdx4IL/dlcache: A simple file-backed download cache

It returns a value pointing to a local copy of the archive, which one can feed to ExternalProject etc.