One library target building both static and shared

Pre-coffee irrational idea, but thought why not put it out there for discussion anyway.

A single library target can be built as static or shared, but not both. This presents problems for some consumers who specifically need one or the other. It gets more complicated when the consumer itself might have switchable behavior which can select whether it wants to consume static or shared libs. Projects that want to support such flexibility have to create two separate targets, one for the static lib and another for the shared lib. That leads to different targets appearing as dependencies for the consumer, which can in turn complicate things like installing, exporting, packaging, etc.

While it is possible to find a way through the above scenario and others like it, I’m wondering if we could do better. Could we have a boolean target property that specified to build both static and shared versions of a target? That would definitely have lots of consequences, but let’s spit-ball it and see how far we get.

  • Consumers could have a target property that selects whether they prefer to link to static or shared libs, where there is a choice. This should be on the consumer, not on the provider.
  • I suspect we would still want to designate such libraries as having their TYPE target property set to SHARED_LIBRARY to avoid breaking user projects. It’s just that the target can also provide a static library. The requirements on a shared library are likely to be a more constrained set than static (e.g. the need for position-independent code). In other words, consider it a shared library, but with extra build artefacts for static. We’d probably have to add some new target properties to support these “extra” things.
  • We may be able to use the one set of object files for both. That could be a really nice feature, saving duplication.
  • A single install(TARGETS) rule could install both the shared and static library bits. Not sure how we’d deal with Windows and the import lib for the DLL clashing with the static lib. That’s an open question for discussion.

I’m sure there are a lot more consequences, but let’s see what discussions the above triggers.

2 Likes

One obvious consequence and problem is: how is it consumed?
If a target can be a shared and a static library at once, how the consumer selects the type he wants?

In a typical project, for example, we want to be able to generate two executables: one dynamically linked and one statically linked. So, it is required to add a level of complexity to command target_link_libraries() to handle that.

1 Like

I personally have seen 2 different cases:

  • As you mention, providing both static and shared for consumer purposes ( w/ hidden symbols)
    • Should the static library limit the public symbols to be the same as the ones from the shared library ? While it does not really matter on Unix-like systems it is, I believe, not feasible on windows ?
    • Does this mean we’ll need a suffix on the static lib or import lib on windows ?
    • Where do symbols go ? If we put the suffix on the staticlib it would be ok but otherwise we might have name clashes
    • What if a package manager maintainer want to install only one of the two ?
    • What about dependencies of such targets ?
      • Usually when building a shared lib, you want to link shared libs, and for static, link static ones
  • For internal use, with full symbol visibility. In this case (I think it’s still better to decouple targets in this case)
    • Needed for unit testing and benchmarking unless you embed those in your DLL (only doctest seems to make this feasible)
    • This means you need to access dependencies that would be marked PRIVATE for the shared lib, but PUBLIC for the static one

And I’m pretty sure there’s more to it than the questions we raised here :slight_smile:

This is all fine, but I think projects should opt into this behavior even being available (that is, it is not completely up to end-users whether a project even supports this feature meaningfully). Quite a number of projects just do:

target_compile_definitions(mytgt PRIVATE BUILT_SHARED=$<BOOL:${BUILD_SHARED_LIBS}>) # or using `if()` for the bool conversion

I have no idea how to write a policy for that. Worse is the configure_file() which contains the information in a public header. That header now needs duplicated for shared/static and interface usage requirements updated to make that Just Work. We’d need to instead suggest:

target_compile_definitions(mytgt PRIVATE # or INTERFACE?
  BUILT_SHARED=$<STREQUAL:$<TARGET_PROPERTY:TYPE>:SHARED>)

Which is…cumbersome to say the least.

Other projects, like VTK and ParaView, do completely different things for shared vs. static builds in some corners of their build system (e.g., with Python module packaging). Plugin handling is also remarkably different. Kit builds are also interesting because the object libraries that go into the actual libraries would need to now be compiled differently.

One convention is that DLL import libraries are named foo.lib. Static libraries are libfoo.lib.

FWIW, I never really liked projects that supported both shared and static in a single build myself. “Pick a lane” is my reaction I guess. My main concern is with:

find_package(Common) # shared and static
find_package(UsesCommon) # uses it statically

add_library(mytgt)
target_link_libraries(mytgt
  UsesCommon # Brings in Common via static linking, but is itself shared
  Common) # Do I really have a choice on `Common` here?
          # Either way is prone to duplicate symbols and if Common
          # isn't ready, I probably get fun "spooky linker issues".

This can already be achieved using OBJECT libraries.

CMake has always modeled “one target == one set of artifacts”. I think changing that will introduce too much complexity. Every place that one can name a target now would need some new way name the target along with an indication of which variant to use.

A possible real world use case for such kind of feature would be in conda-forge packaging. According to CFEP-18, by default packages contain shared library, but if necessary static libraries can be packaged in separate packages named <pkg>-static.

If both <pkg> and <pkg>-static are installed, downstream CMake consumers do not have a way to opt for static libraries, so users of static libraries typically hard-code them (see for example mamba/CMakeLists.txt at 0.10.0 · mamba-org/mamba · GitHub).

However it would be necessary to have some mechanism for which the CMake config files for both variants can be installed independently (similar to what happens on Debug or Release configurations in Multi-Config generators).

Linux distros have similar problems since some packages do get built in shared and static. CMake generally prefers to link libraries via full path anyways, so -l being ambiguous as to which gets picked is not the place to fix it. Instead, find_library might have some top-level control for whether to prefer finding static or shared libraries. CMake configuration files would need some other way to prefer finding a static or shared package though.

Perhaps a good place to start would be some official documentation and best practices on how to do this today, and then figure out how to take steps toward supporting some of the more common use cases. Our team has to deliver a number of libraries in both their static and shared form, and have had to roll our own system to do so, so we would love to see this more officially supported.

For our use cases, the most prevalent issues we run into are differences between MSVC and GNU compilers. With GNU, our wrapper functions can generate object libraries that enable PIC and are shared between the static and shared libraries so we only compile the source once. MSVC, however, requires the object files to be linked to either the shared or static runtimes, so we have to generate two parallel object libraries that are linked to their respective static or shared libraries. There are some other slight differences in compile definitions and options as well, but that seems doable with some new generator expressions that understand the type of a library.

This is one difference. In our projects, static builds typically still use the shared runtime. Assuming static up and down the stack is an extension to the static/shared decision within a single project.