Can't export top level lib with dependency on subdir lib

Could you please advise on this issue. I found similar issue to one that has been posted as “Cannot build sub-directory library if top-level CMakeLists.txt call install(EXPORT …)” but was not answered at https://cmake.org/pipermail/cmake/2018-March/067196.html and not really answered at https://stackoverflow.com/questions/49254613/cannot-build-library-in-subdirectory-when-top-level-cmakelists-txt-calls-install
Consider directory structure as the following

applib (top level SDK library )
     |-- top-level CMakeLists.txt (project(top...)
	 |-- sublibTop project(sublibTop...
	 |   `-- CMakeLists.txt
     |                                     
     |-- sublib (project (sublib .. )
     |   `-- CMakeLists.txt
     |..

If subdir target sublib is included into a list of targets (e.g. install(TARGETS sublibTop sublib … ), then cmake fail during build due to the sublib target is not found at sublibTop level directory (target is built ok but in sub-dir) with error:
install TARGETS given target " sublib" which does not exist in this directory.

However if sublib is not included in the list of install(TARGETS, the error is

CMake Error: install(EXPORT “sublibTop” …) includes target “sublibTop” which requires target “sublib” that is not in the export set.

Top CMakeLists.txt
add_subdirectory(sublibTop)
add_subdirectory(sublib)

sublibTop CMakeLists.txt
add_library(sublibTop…
target_link_libraries( sublibTop sublib)

install(TARGETS sublibTop sublib
EXPORT projTargets
)
install(EXPORT projTargets

)

sublib CMakeLists.txt
add_library(sublib…
I am looking for a way to use cmake figure out all installation targets and export elements based on target properties and target dependencies and commands for install/export of single top level target.
On clue is that the 1st error does not occur on machine using cmake 3.17 but with 3.11 being used on another machine (Jenkins) but I am not sure it is not caused by the order or build and I prefere not to include sublib into install(TARGETS list since cmake knows it is required for sublibTop .
Thanks

There are a few ways of doing this:

  • use a single EXPORT set in each directory where a target is declared followed by install(EXPORT) at a common ancestor
  • use multiple export sets, one for top and another for sublib.

Thank you Mr. @ben.boeckel, but both these options defeat the purpose of installing by using just the top level library and having CMake to use dependency specified by the target_link_libraries( sublibTop sublib) and process install(TARGETS sublibTop with all its dependencies. I wish this would be a default behavior and if the user does not want dependency lib to be installed they would have to set some option. These 2 listed approaches, if I understand them by the short note correctly , are not better than the current approach I am using in which I have to indicate not just sublibTop but sublib as shown in example above. I am trying to find existing approach or maybe specify CMake enhancement allowing less code and use so - called target-centric approach in which user specified targets and their dependencies and let CMake do the rest including Install. Regards. YS

If sublib is not installed, how is the install tree going to get access to it? The target also needs installed so that CMake knows about it too. CMake can’t automatically add targets to export sets because you end up with duplicate target definitions that way.

Of course I do want to install sublib. It has to be installed. I merely arguing that specifying target_link_libraries( sublibTop sublib)
install(TARGETS sublibTop … without sublib in this list in this list should be sufficient for cmake to conduct such install and add sublib to projTarget list for export bc CMake knows sublibTop depends on sublib. Then install(EXPORT projTargets would include sublib automatically

That’s the second part of my previous comment. It cannot automatically add it to that export set becauseif another target does the same thing in the same project, what happens now?

When installing 2nd target whicch has sublib dependency cmake could easily determine that the 1st target already installing sublib and skip duplicate install. This is much better than having the user to specify sublib in 1st install(TARGET, but not in the 2nd because it makes cmake code cleaner. Plus there might be no 2nd target with the same dependency …

I don’t know how to document that. It feels like it ends up being “spooky action at a distance”. The policy for it is ill-scoped. Does it matter if the policy is set for sublib? sublibTop? What if they disagree? I suppose it could dodge being a policy if it becomes a new option (it would have to default to OFF, flipping the default would certainly be a policy). But then the scope of that variable needs documentation and explanation with the same ambiguity I mentioned earlier. Personally, I want to error if I forget to export a target instead of some magic. We’ve had way too many problems with these kinds of magic behaviors in the past and I don’t like the idea of adding a new one either, default or not. The reason I want an error is that I want to know what export file will have the target, what namespace it will live in, what that export file will be named, etc.

Ok, then how about creating helper function like ones in the CMakePackageConfigHelper module?

installTargetWithItsDepenencies(myTarget, destinationDir, installWithPublicHeaders, ComponentList,…) it would fill in MYTARGETS_LIST variable

The idea is still to streamline install based just on target_link_libraries ()

I just want to add that IMPORTED libraries have to be included as well maybe with another enable/disable flag. I don’t see installation treat them in consistent manner.

The user provides all dependencies in the target_link_libraries why CMake requires ( except known historical reason) to have separate way of installing them .? I think setting something like OPTIONAL flag should be sufficient for CMake install to skip installing if destination already has same target if same version .

Best Regards

Yakov

Imported targets certainly should not be re-exported as part of your project. You need to do the find_package for them from your config file. There’s no way CMake is going to be able to help with this in a generic manner because it doesn’t know:

  • what package to find
  • what version to require on it
  • what components are needed
  • whether it is required

for any given find_package of your project. That will have to be a manual step without something like this logic which is way more rigid of a project structure than CMake works with today. Even that code makes assumptions about how find_package is going to work for the projects it is involved with.

Did you mean that imported library should not be exported and if I am exporting my sublibTop library for consumers, consumers have to use find_package for mine and separately for imported libraries?

Let’s consider case alternative to find_package when xxxFindPackage.cmake does not exist and external dependency is added via add_library(implib, IMPORTED …) set_target_properties(implib PROPERTIES IMPORTED_LOCATION …). target_include_directory( implib INTERFACE …include)

and then linked target_link_libraries(sublibTop sublib implib …) as implib is needed for the sublibTop consumers for building their executables.

So in this case sublibTop and sublib would be installed using install( TARGETS sublibTop sublib … EXPORT subTargets … but implib would have to be install(FILES implib …

This is inconsistent.

As I understand your suggestion, it is applicable when xxxFindimplib.cmake exists, and in my CMakeLists.txt I would have find_package(implib_pkg 0.0.1 REQUIRED) (or find_library(implib …) ) and xxxFindimplib.cmake definitions of target names will be used (implib_LIBRARIES as well as implib_INCLUDE_DIRECTORIES, …) in my taget_link_libraries(sublibTop sublib ${implib_LIBRARIES}… and installed with install(TARGETS sublibTop sublib .? …EXPORT sublibTopTargets.

Then to make my sublibTop library accessible via find_package for my library consumer, I should not include implib targets into install(EXPORT list and rely on consumers to use find_package(implib … in their CMakeLists.txt as well as for my sublibTop library, right?

Then it seems inconsistent since I’d like to provide self contained package (sublibToopConfig.cmake that defines all libs) to my sublibTop consumers. So I would prefer to export all libraries needed to build with my sublibTop library however I cloud list all libs that require find_ package in my installation notes.

Best Regards

Yakov

No, your project’s foo-config.cmake does a find_package(yourdep ...). find_dependency may suffice out if you don’t have components in your package though, but version requirements would need forwarded somehow.

It is my opinion that IMPORTED libraries should only come from find_package. So the error here is using anIMPORTED library outside of that context. You find it at build time and then re-find it when you’re being found (it might be in a different prefix from the build time, an ABI-compatible version bump may have occurred, etc.).

If you’re building sublib, export it. But don’t build it as IMPORTED. Put it in its own EXPORT set, add it to another. Either way works. If you’re using an external sublib, use find_package during the build and again when your package is being found. It’s still self-contained either way.

Thanks, could you please clarify “ use find_package during the build and again when your package is being found. It’s still self-contained either way.”

You meant that find_package statement is being used for te 1st time in my CMakecode that builds and installs my own sublibTop lib and then 2nd time in the code of a consumer of my library , right?

But these find_package would be used with different package names. 1st with the the impLib external to my build and 2nd would use my sublibTop target.

So could you please illustrate this statement.

Also , after I call find_package (implib REQUIRED) I get the libs and include dir which is set by implib’s Find.cmake , I link them with target_ link_libraries.

Is it ok to use add_library on such libs as IMPORTED?

What about install( TARGETS and install(EXPORT for such targets ?

Should libs target obtained via find_package be installed & exported together with my own sublibTop built target or rely my library consumer to find my own lib package and then find package for my dependencies? Note typically I put 3rd party libraries ( with arc and include) in separate directory at the same level as my sublibTop project or even under 3rdParty directory which is at the same dir levels as my project

Let’s say you require Qt5 for your project. In your build you have:

find_package(Qt5 5.9 REQUIRED COMPONENTS Core Widgets)

You end up finding Qt 5.12 during the build. In your mypackage-config.cmake, you then have:

include(FindDependencyMacro)
find_dependency(Qt5 5.12 COMPONENTS Core Widgets)

You bump the minimum required up because although you needed 5.9 to build, once you built against 5.12, you need at least that version (in general). For Boost, you’d need EXACT for the version you built against because it has no ABI guarantees between version numbers. Other packages may have different allowable relationships.

All consumers need to do is find_package(mypackage) because your package finds Qt5 as needed during that process.

If you support an external sublib, I suggest something like:

option(USE_EXTERNAL_SUBLIB "blah blah" OFF)
add_library(mysublib INTERFACE)
if (USE_EXTERNAL_SUBLIB)
  find_package(sublib ...)
  target_link_libraries(mysublib INTERFACE sublib_imported_target)
  # tell mypackage-config.cmake to add code to find sublib with any required version info
else ()
  add_subdirectory(sublib)
  target_link_libraries(mysublib INTERFACE sublib_built_target)
  # sublib_built_target needs exported with your project now (probably done inside the subdir), but `mypackage-config.cmake` needs to include its export file somehow
endif ()
# export mysublib

How would I do "````# tell mypackage-config.cmake to add code to find sublib with any required version info
``" ?

Should I include

find_package(sublib ..) into template [mypackage-config.cmake.in](http://mypackage-config.cmake.in)  file?

(Reference: https://cmake.org/cmake/help/latest/module/CMakePackageConfigHelpers.html#generating-a-package-configuration-file )

For theTRUE case in

if (USE_EXTERNAL_SUBLIB)

I don’t see how IMPORTED been used at all. In which case would t be used?

Also could you please add directions for install(TARGET and install(EXPORT to clarify if lib found by find_package or IMPORTED

Also this whole approach is suitable only if the external lib sublib supports find_package() so its distribution has sublibFind.cmake. In my initial problem description I assumed there none.
As cmake does not allow to install IMPORTED libs as targets the only way I see it is done by install(FILES which makes install inconsistent

Best Regards

yakov

Global properties, INTERNAL cache variables, or setting directory properties on the top-level directory. I prefer the former, but any works.

No, find_dependency is preferred unless you’re doing COMPONENT logic for your package.

The IMPORTED target comes from find_package(sublib).

The subllb directory in your project would export the target. You’d also install and export your mysublib interface target. It’s just like any other target in that respect.

Yes. You can ship your own Findsublib.cmake with your project too. But if there isn’t one and you don’t support an external copy, you’ll only ever use the else() branch in the code I showed and you can probably skip mysublib.

IMPORTED libraries come from outside your project. There are no files for you to install for them.

Thus there is a contradiction as from one hand you indicated to for external libraries which i would use find_package (sublib…) :

“… It’s just like any other target in that respect.” So after find_ package succeeds with sublib_FOUND true, I should install( TARGET sublib … and install(EXPORT it . While from the other handle sublib would be IMPORTED target as find_package ( sublib) would set it as IMPORTED as per “ IMPORTED libraries come from outside your project. There are no files for you to install for them.“ the files to install in this context are sublib.so or .a which are there and could be under my to level project directory.

So I am still not clear what is the best practice for external sublibs not built with CMake but needed as dependencies for my own library and so needed by the build to be done consumers of my library.

Consider case of external sublib library ( eg Postgres pqxx ) which has no its own. pqxxConfig.cmake but .so, .a and headers are installed on the host into directory under my project dir.

You do not install imported targets. End of story. You find_package them again if you need the imported targets later.

If you have an external project which doesn’t have its own config.cmake file, write a FindFoo for it and install it with your project. You still use find_package.

  1. ok, just to clarify, if target is being found by find_package, it should not be installed nor exported (as it is IMPORTED). Since my library dependent on IMPORTED target, the consumers of my library has to use find_package to find (=IMPORT) in their CMake files. To support find_file I would need to write FindFoo (in this context FindimpSublib.cmake like for example FindPQ.cmake for PostgreSQL …) and include it into my package.
    I don’t need to write FooConfig.cmake (foo-config.cmake), just one FindFoo.cmake. Right?
    Then is there a recommended location in the project tree for this file and in the package file? I assume it should be under my library and not under the imported target tree .

  2. What if external packages does not provide FooConfig.cmake but provides foo.pc file? Should not then pkg_search_module /pkg_check_module be used instead of find_package /writing FooFind.cmake? What are pro and contra points?

I