Pre-coffee irrational idea, but thought why not put it out there for discussion anyway.
A single library target can be built as static or shared, but not both. This presents problems for some consumers who specifically need one or the other. It gets more complicated when the consumer itself might have switchable behavior which can select whether it wants to consume static or shared libs. Projects that want to support such flexibility have to create two separate targets, one for the static lib and another for the shared lib. That leads to different targets appearing as dependencies for the consumer, which can in turn complicate things like installing, exporting, packaging, etc.
While it is possible to find a way through the above scenario and others like it, I’m wondering if we could do better. Could we have a boolean target property that specified to build both static and shared versions of a target? That would definitely have lots of consequences, but let’s spit-ball it and see how far we get.
Consumers could have a target property that selects whether they prefer to link to static or shared libs, where there is a choice. This should be on the consumer, not on the provider.
I suspect we would still want to designate such libraries as having their TYPE target property set to SHARED_LIBRARY to avoid breaking user projects. It’s just that the target can also provide a static library. The requirements on a shared library are likely to be a more constrained set than static (e.g. the need for position-independent code). In other words, consider it a shared library, but with extra build artefacts for static. We’d probably have to add some new target properties to support these “extra” things.
We may be able to use the one set of object files for both. That could be a really nice feature, saving duplication.
A single install(TARGETS) rule could install both the shared and static library bits. Not sure how we’d deal with Windows and the import lib for the DLL clashing with the static lib. That’s an open question for discussion.
I’m sure there are a lot more consequences, but let’s see what discussions the above triggers.
One obvious consequence and problem is: how is it consumed?
If a target can be a shared and a static library at once, how the consumer selects the type he wants?
In a typical project, for example, we want to be able to generate two executables: one dynamically linked and one statically linked. So, it is required to add a level of complexity to command target_link_libraries() to handle that.
As you mention, providing both static and shared for consumer purposes ( w/ hidden symbols)
Should the static library limit the public symbols to be the same as the ones from the shared library ? While it does not really matter on Unix-like systems it is, I believe, not feasible on windows ?
Does this mean we’ll need a suffix on the static lib or import lib on windows ?
Where do symbols go ? If we put the suffix on the staticlib it would be ok but otherwise we might have name clashes
What if a package manager maintainer want to install only one of the two ?
What about dependencies of such targets ?
Usually when building a shared lib, you want to link shared libs, and for static, link static ones
For internal use, with full symbol visibility. In this case (I think it’s still better to decouple targets in this case)
Needed for unit testing and benchmarking unless you embed those in your DLL (only doctest seems to make this feasible)
This means you need to access dependencies that would be marked PRIVATE for the shared lib, but PUBLIC for the static one
And I’m pretty sure there’s more to it than the questions we raised here
This is all fine, but I think projects should opt into this behavior even being available (that is, it is not completely up to end-users whether a project even supports this feature meaningfully). Quite a number of projects just do:
target_compile_definitions(mytgt PRIVATE BUILT_SHARED=$<BOOL:${BUILD_SHARED_LIBS}>) # or using `if()` for the bool conversion
I have no idea how to write a policy for that. Worse is the configure_file() which contains the information in a public header. That header now needs duplicated for shared/static and interface usage requirements updated to make that Just Work. We’d need to instead suggest:
target_compile_definitions(mytgt PRIVATE # or INTERFACE?
BUILT_SHARED=$<STREQUAL:$<TARGET_PROPERTY:TYPE>:SHARED>)
Which is…cumbersome to say the least.
Other projects, like VTK and ParaView, do completely different things for shared vs. static builds in some corners of their build system (e.g., with Python module packaging). Plugin handling is also remarkably different. Kit builds are also interesting because the object libraries that go into the actual libraries would need to now be compiled differently.
One convention is that DLL import libraries are named foo.lib. Static libraries are libfoo.lib.
FWIW, I never really liked projects that supported both shared and static in a single build myself. “Pick a lane” is my reaction I guess. My main concern is with:
find_package(Common) # shared and static
find_package(UsesCommon) # uses it statically
add_library(mytgt)
target_link_libraries(mytgt
UsesCommon # Brings in Common via static linking, but is itself shared
Common) # Do I really have a choice on `Common` here?
# Either way is prone to duplicate symbols and if Common
# isn't ready, I probably get fun "spooky linker issues".
This can already be achieved using OBJECT libraries.
CMake has always modeled “one target == one set of artifacts”. I think changing that will introduce too much complexity. Every place that one can name a target now would need some new way name the target along with an indication of which variant to use.
A possible real world use case for such kind of feature would be in conda-forge packaging. According to CFEP-18, by default packages contain shared library, but if necessary static libraries can be packaged in separate packages named <pkg>-static.
If both <pkg> and <pkg>-static are installed, downstream CMake consumers do not have a way to opt for static libraries, so users of static libraries typically hard-code them (see for example mamba/CMakeLists.txt at 0.10.0 · mamba-org/mamba · GitHub).
However it would be necessary to have some mechanism for which the CMake config files for both variants can be installed independently (similar to what happens on Debug or Release configurations in Multi-Config generators).
Linux distros have similar problems since some packages do get built in shared and static. CMake generally prefers to link libraries via full path anyways, so -l being ambiguous as to which gets picked is not the place to fix it. Instead, find_library might have some top-level control for whether to prefer finding static or shared libraries. CMake configuration files would need some other way to prefer finding a static or shared package though.
Perhaps a good place to start would be some official documentation and best practices on how to do this today, and then figure out how to take steps toward supporting some of the more common use cases. Our team has to deliver a number of libraries in both their static and shared form, and have had to roll our own system to do so, so we would love to see this more officially supported.
For our use cases, the most prevalent issues we run into are differences between MSVC and GNU compilers. With GNU, our wrapper functions can generate object libraries that enable PIC and are shared between the static and shared libraries so we only compile the source once. MSVC, however, requires the object files to be linked to either the shared or static runtimes, so we have to generate two parallel object libraries that are linked to their respective static or shared libraries. There are some other slight differences in compile definitions and options as well, but that seems doable with some new generator expressions that understand the type of a library.
This is one difference. In our projects, static builds typically still use the shared runtime. Assuming static up and down the stack is an extension to the static/shared decision within a single project.
I wrote a blog post on my (and thus Halide’s) solution to this problem. You can read it here and see the code here, but I’ll summarize below.
In my experience, a single build is far more likely to require one library type or the other, not both. Even when a single build does require both, it is easy enough to split the targets into two directories and call find_package twice: once with variables set to find the static libraries, and once in the other directory for the shared libraries.
Ultimately, it seems the most common reason people want to build both at once is to simplify distributing packages that contain both. There are many ways CMake could improve this story without complicating the target model:
There could be a standard way to hint to the find_* commands that static or shared libraries should be preferred. The <PkgName>_ROOT variable affects find_* calls nested inside a find_package call; there could be a similar <PkgName>_SHARED_LIBS variable that would cause shared libs to be preferred.
To keep BUILD_SHARED_LIBS and <PkgName>_SHARED_LIBS similar in function and user-settable, find_package could be augmented with STATIC and SHARED keywords that enforce one or the other for projects that care.
The install() command is already aware of build type (ie. Debug, Release) differences. Perhaps it and the configure_package_config_file commands could be co-evolved to support library type as well. The latter command would provide macros for loading the correct generated CMake files.
CPack could provide a command line interface for installing multiple projects into the same package (ie. what CPACK_INSTALL_CMAKE_PROJECTS does now).
There’s still a question about whether these settings should constitute a requirement or a preference and what the defaults should be. Here are my thoughts on that:
If find_package is called with STATIC or SHARED, this should be considered a requirement and the call will fail-to-find if the requested type isn’t there. Supplying both is an error.
The <PkgName>_SHARED_LIBS variable, if set, should be considered a requirement.
If neither is set, then to match the build interface (ie. FetchContent and add_subdirectory), the BUILD_SHARED_LIBS variable should be consulted as a preference. If only one type is available, it will be loaded (potentially with a warning?).
The find_package keywords override the <PkgName>_SHARED_LIBS variable, which overrides BUILD_SHARED_LIBS.
This convention degrades gracefully: if the new variables/keywords aren’t set, then it behaves the same way as before. It also keeps the build and install interfaces consistent, as long as <Pkg>_SHARED_LIBS is checked by the project() command or something to override BUILD_SHARED_LIBS in the build interface, too (this could also be implemented in add_*, but it is nicer to if-test only for BUILD_SHARED_LIBS).
The existing Find modules are all over the place with respect to this issue:
FindBLAS and FindLAPACK use BLA_STATIC.
FindBoost, FindOpenSSL, FindProtobuf, and FindPython(2/3) use <Pkg>_USE_STATIC_LIBS.
FindCUDA (deprecated) used CUDA_USE_STATIC_CUDA_RUNTIME.
FindCUDAToolkit uses targetsCUDA::<target>_static and CUDA::<target> (for shared).
FindGLEW uses GLEW_USE_STATIC_LIBS, but only to optionally include GLEW::glew_s. GLEW::GLEW is defined to be either that one or GLEW::glew (shared), but the docs don’t make it clear which it is if both are present.
FindHDF5 uses HDF5_USE_STATIC_LIBRARIES.
FindPkgConfig creates variables with _STATIC in them, but it’s not clear which the IMPORTED_TARGET switch for pkg_check_modules prefers.
FindwxWidgets uses wxWidgets_USE_STATIC.
I think that having simultaneous targets for static/shared is a mistake in general because it forces the consumer to make a decision that cannot be overridden. If you just link to Boost::<component>, a user may set Boost_USE_STATIC_LIBS as a cache variable. No such luck with CUDA.
I also think that BUILD_SHARED_LIBS and the pseudo-convention of preferring shared libs for Find modules are inconsistent. The advent of FetchContent suggests to me that <Pkg>_SHARED_LIBS is the better convention, but I welcome debate on this point.
