Exporting and packaging prebuilt libraries in cmake

Le_MonSiC · August 27, 2021, 12:40pm

instead of asking a question directly, I’ll expose my use case and the way I tried (but failed) to solve it.

Say I have:

3 shared libraries A, B and C
A require B and C
A comes with a set of headers

That’s it, no extra information, it’s provided by a vendor and not possibly subject to any change (modernization/cmake packages, etc).
A should always be packaged with B and C. I should only need to link with A and cmake should transitively link with B and C.

Now, I’d like to make it more “modern cmake” friendly and by able to:

First usecase: Create a repo containing these libs and calling add_subdirectory() from a parent project.
First usecase: Create a package (say debian pkg . deb) containing the relevant AConfig.cmake AConfigVersion.cmake and ATargets.cmake. Then a simple system install of the pkg and a find_package() should to the trick.

What has been done:

I tried using INTERFACE IMPORTED library and INTERFACE.

Because I want to support packaging the libs INTERFACE IMPORTED can’t be used (you can’t install it as far as I know/tested).

INTERFACE is working fine for the first usecase, using add_subdirectory(), headers are found, everything links, but because the user may not have at this point, the shared lib in is path, he can’t run the tests for instance.

Then comes the export part needed to make the shared libs available and to make find_package() work. I succeed to export/package the libs A B C, the headers for A and the files needed for find_package().

But when in a client library D, find_package(A REQUIRED) finds the lib (no messages such as "Could not find a package configuration file provided by “A” ") it doess NOT create any target I can link on. I took a look at the generated ATargets.cmake:

# generated stuff before

add_library(A::A INTERFACE IMPORTED) # This is cannot be used at all, does not create a target I can link on unless I remove IMPORTED

set_target_properties(A::A PROPERTIES
  INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include" # that's fine
  INTERFACE_LINK_LIBRARIES "A.so;B.so.1.0;C.so.1.0" # that's not fine I need ${_IMPORT_PREFIX}/${CMAKE_INSTALL_LIBDIR}/ before each libs like it did for the headers
)

# generated stuff after

Note that if I remove the IMPORTED in the add_library of the ATargets.cmake and remove the namespace stuff, the target A is correctly accessible in any client cmake project using find_package but it’ll NOT link correctly probably because A.so (and B and C) is not referenced using ${_IMPORT_PREFIX}/lib/A.so. I tried adding ${_IMPORT_PREFIX}/lib/ before all libs in INTERFACE_LINK_LIBRARIES, removed the IMPORTED keyword and namespace stuff and guess what, it works perfectly… Now, how do I do that without the trick.

The content of AConfig.cmake.in is:

@PACKAGE_INIT@
include(CMakeFindDependencyMacro)

# Add the targets file
include("${CMAKE_CURRENT_LIST_DIR}/ATargets.cmake")

# check_required_components(@PROJECT_NAME@) # It is commented because unless I apply the fix specified earlier (IMPORTED and namespace removed), during the find_package call I get:
#################
# CMake Error at /usr/lib/cmake/A/AConfig.cmake:8 # (check_required_components):
#   Unknown CMake command "check_required_components".
#################

That’s pretty much it, the rest of this post is implementation details. This code below can be used to solve use case 1 but the export package and cmake config/target files are not correct.

add_library(A
            INTERFACE)

add_library(A::A ALIAS A)


target_include_directories(A
                           INTERFACE "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>"
                                     "$<INSTALL_INTERFACE:include>")

target_link_libraries(A
                      INTERFACE "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/lib/A.so>"
                                "$<INSTALL_INTERFACE:A.so>"
                                "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/lib/B.so>"
                                "$<INSTALL_INTERFACE:B.so>"
                                "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/lib/C.so>"
                                "$<INSTALL_INTERFACE:C.so>")

#### install

install(TARGETS  A
            EXPORT   ATargets
            RUNTIME  DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT A_Runtime
            LIBRARY  DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT A_Runtime NAMELINK_COMPONENT A_Development
            ARCHIVE  DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT A_Development)

    install(DIRECTORY   "lib/"
            DESTINATION ${CMAKE_INSTALL_LIBDIR})

    install(DIRECTORY   "${CMAKE_CURRENT_SOURCE_DIR}/include"
            DESTINATION "include")

    write_basic_package_version_file(AConfigVersion.cmake
                                    VERSION "${PACKAGE_VERSION}"
                                    COMPATIBILITY SameMajorVersion)

    install(EXPORT      ATargets
            FILE        ATargets.cmake
            NAMESPACE   A::
            DESTINATION "lib/cmake/A")

    configure_file(AConfig.cmake.in AConfig.cmake @ONLY)
    install(FILES       "${CMAKE_CURRENT_BINARY_DIR}/AConfig.cmake"
                        "${CMAKE_CURRENT_BINARY_DIR}/AConfigVersion.cmake"
            DESTINATION "lib/cmake/A")

# Then some cpack stuff that is not affecting the work done earlier

In a client lib/cmake project after the installation of the generated package (generated by cpack):

find_package(A)

target_link_libraries(my_project PUBLIC/PRIVATE A::A) # Should bring in the headers and link with A B and C

Documentation:
I can put only to 2 links… why ?

check pastbin then: https://pastebin.com/raw/K2KuNF5L

Thanks for reading/helping

klirichek · August 31, 2021, 11:26am

I’ve solved similar task with custom location of prebuilt openssl.
It looks a bit similar to your model: openssl has library + dependency from crypto.
I didn’t used full-stack cmake project, but instead made one dedicated OpenSSLConfig.cmake (since actually all the rest - XXX-version, XXX-targets - are just helpers, they’re good when automatically generated, but seems redundant when manually written.
(and code here is for windows. It is a bit more complex, as it has separate implib apart from just so/dll)

# Compute the installation prefix relative to this file.
get_filename_component (_IMPORT_PREFIX "${CMAKE_CURRENT_LIST_FILE}" PATH)
set (_IMPORT_PREFIX "${_IMPORT_PREFIX}/OpenSSL-Win64")
set (MSRT MD)
set (_ssl "libssl64")
set (_cr "libcrypto64")
set (_ssldll "libssl-1_1-x64")
set (_crdll "libcrypto-1_1-x64")
...
	# Create imported target OpenSSL::SSL shared
	add_library (OpenSSL::SSL SHARED IMPORTED)
	set_target_properties (OpenSSL::SSL PROPERTIES
			INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
			INTERFACE_LINK_LIBRARIES "OpenSSL::Crypto"
			IMPORTED_CONFIGURATIONS "RELEASE;DEBUG"
			IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C"
			IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C"
			IMPORTED_IMPLIB_DEBUG "${_IMPORT_PREFIX}/lib/VC/${_ssl}${MSRT}d.lib"
			IMPORTED_IMPLIB_RELEASE "${_IMPORT_PREFIX}/lib/VC/${_ssl}${MSRT}.lib"
			IMPORTED_LOCATION "${_IMPORT_PREFIX}/${_ssldll}.dll"
			MAP_IMPORTED_CONFIG_RELWITHDEBINFO Release
			MAP_IMPORTED_CONFIG_MINSIZEREL Release
			)

	# Create imported target OpenSSL::Crypto shared
	add_library (OpenSSL::Crypto SHARED IMPORTED)
	set_target_properties (OpenSSL::Crypto PROPERTIES
			INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
			IMPORTED_CONFIGURATIONS "RELEASE;DEBUG"
			IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C"
			IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C"
			IMPORTED_IMPLIB_DEBUG "${_IMPORT_PREFIX}/lib/VC/${_cr}${MSRT}d.lib"
			IMPORTED_IMPLIB_RELEASE "${_IMPORT_PREFIX}/lib/VC/${_cr}${MSRT}.lib"
			IMPORTED_LOCATION "${_IMPORT_PREFIX}/${_crdll}.dll"
			MAP_IMPORTED_CONFIG_RELWITHDEBINFO Release
			MAP_IMPORTED_CONFIG_MINSIZEREL Release
			)

Acrually here is only some of properties are relevant - INTERFACE_INCLUDE_DIRECTORIES provides path to headers, IMPORTED_LOCATION is path to dll, and INTERFACE_LINK_LIBRARIES provides that “A requires B” - i.e. if you link with OpenSSL, then Crypto also will be in game. IMPLIB is purely win-specific and not necessary on so-based systems. And specific configs routes between Debug and all kind of Release variants. Sure, it might be also written using full route - with configuration/generation/whatever, but actually the purpose is one simple thing - make prebuilt library from aside available in the project without intervention inside (this file OpenSSLConfig.cmake is placed aside the folder with openssl, so no intervention inside performed). So, for me it looks much simpler to investigate result of an existing export, and then repeat similar thing. Result is very plain and simple - no logic, just set of properties.

Le_MonSiC · August 31, 2021, 6:50pm

Thanks for your answer, I have similarly ended with a custom LibConfig.cmake. It works fine, as expected. Still, I’m disappointed that we are not able to “construct” a library from bits and pieces found on the disk, like some .a, .so /.lib .dll and headers using only cmake standard interfaces and export/packaging functionalities (without hand rolling a set of Config/Target cmake files). It may be possible but I have not found anything and the documentation of cmake lacks examples for edge case, to say the least.
The idea was to create the Config.cmake and the ConfigVersion.cmake the traditional way and hand roll a FindLib.cmake in a subdir of the directory containing the config.cmake and ConfigVersion.cmake.
FindLib.cmake use the old find_path/find_library to assemble a target from scratch using find_package_handle_standard_args, add_library(lib::lib SHARED/STATIC IMPORTED) and a set of IMPORTED_LOCATION (for release/debug etc.) and INTERFACE_* attributes.

Works fine, crappy, cmake should provide a way to do that without dirtying hands… Still cmake is very useful and there is no viable alternative and works fine most of the time.

klirichek · September 1, 2021, 6:17am

I think, rolling FindLib to use LibConfig is a bit messing.
FindXXX literaly searches, and that is often more expensive.
LibConfig, in turn, knows exactly it’s binaries.
Both finishes providing targets for usage. But finds usually provides ‘unknown’ library, and Config may provide known (static, shared, whatever), because it knows.

The rest (creation of imported target and setting of their properties) are same in both approaches.
In our project mentioned SSL, with bunch of another 3-rd party prebuilt libs are placed in special bundle. I’ve just added XXXConfig.cmake for each of them - all are like the one about OpenSSL I’ve cited. For the rest we use uniform FindXXX calls. But on Windows, if the bundle is present, we have couple of lines:

LIST (APPEND CMAKE_PREFIX_PATH "${LIBS_BUNDLE}")
set (CMAKE_FIND_PACKAGE_PREFER_CONFIG TRUE)

From that point all works seamless of the system. That is - on Windows with bundle find_package actually first looks in the bundle and uses XXXConfig scripts. And if they’re not available - it falls back to FindXXX, into more expensive exploring and searching.