-Wl,-force_link flags not working correctly

Code
1

I have a very strange issue which I suspect is a cmake bug. I’ve tried to reproduce with a small example but the small example works fine. I’ve got a multi-directory project. In one of those projects I have a CMakeLists.txt that includes:

find_library(CoreMedia_cmake_link CoreMedia)
target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC ${CoreMedia_cmake_link})
...
target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  -Wl,-force_load avfilter)

Another executable then tries to link against the utils_video_encode_decode_ffmpeg library via:

target_link_libraries(ml_auto_edit PRIVATE utils_video_encode_decode_ffmpeg)

If I generate a makefile and run VERBOSE=1 make I see the following linker command (omitted stuff that’s irrelevant):

cd /Users/oliverdain/Documents/code/revl/cpp/build/build/OSX/Debug/ml/auto_edit && /usr/local/Cellar/cmake/3.15.5/bin/cmake -E cmake_link_script CMakeFiles/ml_auto_edit.dir/link.txt --verbose=1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14

...

../../utils/video/encode_decode_ffmpeg/libutils_video_encode_decode_ffmpeg.a -Wl,-force_load -framework CoreMedia /Users/oliverdain/Documents/code/revl/.install/OSX/Debug/ffmpeg/3.4.2.r12/lib/libavfilter.a 

...

Note that the -Wl,-force_load comes before -framework CoreMedia instead of coming before avfilter as it should.

I realize that you’ll want to see the full CMakeLists.txt files but there’s a ton of them and they’re quite large. As I said above, I’ve tried to create a small stand-alone example that reproduces the issue but it works as expected. So I’m hoping somebody might know how this kind of thing could happen in general so I’d have a few things to check out. Or, perhaps it is a cmake bug and there’s enough info here to debug it.

Thanks!

2

I’m not an expert on target_link_libraries at all, but my naive understanding lets me think that:

target_link_libraries(<target> PUBLIC a b)

is equivalent to:

target_link_libraries(<target> PUBLIC a)
target_link_libraries(<target> PUBLIC b)

I guess you need to “keep together” -Wl,-force_load and avfilter by surrounding them with double-quotes:

target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  "-Wl,-force_load avfilter")
3

in your example avfilter gets expanded to a full path to static lib.
Is avfilter a target part of your build or is it an IMPORTED TARGET of some sort?

as noted by @McMartin your statement:

target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  -Wl,-force_load avfilter)

consider the two arguments -Wl,-force_load and avfilter to be 2 differents targets, one being passed as is because it’s a link flags and the other one treated as a target known to CMake (plain or imported).

See: https://cmake.org/cmake/help/latest/command/target_link_libraries.html
in order to check the rule of TLL ordering done by CMake.
as stated there you may have to tweak expected correct order with TLO:
https://cmake.org/cmake/help/latest/command/target_link_options.html

4

To avoid any misinterpretation of your requirements, I suggest two things:

  1. Because it is a link option, not a library, use preferably command target_link_options.
  2. Use LINKER: prefix to ensure correct formatting for your linker tool.

So, you have:

target_link_options (utils_video_encode_decode_ffmpeg PUBLIC "LINKER:-force_load,avfilter")
3 Likes
5

Thanks all for the suggestions. I’m making progress. The library is indeed an imported static library defined as so in my root CMakeLists.txt:

add_library(avfilter STATIC IMPORTED GLOBAL)
set_target_properties(avfilter
PROPERTIES
IMPORTED_LOCATION /Users/oliverdain/Documents/code/revl/.install/OSX/Debug/ffmpeg/3.4.2.r12/lib/libavfilter.a
INTERFACE_LINK_LIBRARIES "x264;z;pthread;m;avresample;swscale;postproc;avcodec;swresample;avutil;dl;"
)

If I change the target_link_libraries bit to

target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  "-Wl,-force_load /Users/oliverdain/Documents/code/revl/.install/OSX/Debug/ffmpeg/3.4.2.r12/lib/libavfilter.a")

it works as expected. However I have to repeat the full path. Doing

target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  "-Wl,-force_load avfilter")

causes the link line to contain -Wl,-force_load avfilter rather than the expanded path to avfilter. So I tried

get_target_property(avfilter_path avfilter IMPORTED_LOCATION)

but then I get

get_target_property() called with non-existent target "avfilter"

I don’t understand why get_target_property doesn’t know about the target when it clearly exists. Since it’s defined in the root cmake I belive it should be available in the sub-project but it appears that it is not.

6

I figured it out! The imported target was defined in the root cmake file after the subdirectories were imported so the property wasn’t available. Changing the order or using a generator expression like

target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC  "-Wl,-force_load $<TARGET_PROPERTY:avfilter,IMPORTED_LOCATION>")

did the trick.

Thanks to all who helped!

7

Ug. I spoke too soon.

There’s actually several libraries I’m trying to -force_load. As you can see, avfilter for example, depends on avresample, swscale, etc. and this is defined in the set_target_properties. If I link them like I was doing before cmake ensures that things are linked in the correct order. However, when I put them in quotes with the generator expression cmake no longer knows that these refer to the libraries I’d defined earlier so it doesn’t not ensure correct link order and I get undefined symbols at link time. I’ll try the target_link_options path instead and see if that does it.

8

target_link_options doesn’t seem to work. It puts the linker flag just once at the beginning of the link line rather than applying it to the library.

9

I got it down to a simpler example. I decided to move this discussion to StackOverflow as that’s generally more active. You can find it here: https://stackoverflow.com/questions/59204159/force-linked-static-libraries-with-cmake-not-working

10

I guess that’s because you need both:

  • target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC avfilter) for the “regular” dependency
  • target_link_libraries(utils_video_encode_decode_ffmpeg PUBLIC "-Wl,-force_load $<TARGET_PROPERTY:avfilter,IMPORTED_LOCATION>") for the “force load”

Putting them together:

target_link_libraries(utils_video_encode_decode_ffmpeg
  PUBLIC
    avfilter
     "-Wl,-force_load $<TARGET_PROPERTY:avfilter,IMPORTED_LOCATION>"
)
11

Thanks for the suggestion Alain. I think that will work most of the time but it’s not guaranteed to work. Let’s assume avfilter depends on some other library like math or bzip. cmake will re-order the “plain”, not-force linked avfilter linking to ensure that avfilter comes before bzip on the link line so that the symbols it needs to bzip get linked. But nothing is using avfilter so the linker drops all its symbols when linked in the regular way. Then later we might have the -Wl,-force_load link but that would come after bzip which means we’d get a failed link. In short "-l/usr/lib/avfilter -lbzip -Wl,-force_load -l/usr/lib/avfilter` doesn’t work and there’s nothing to guarantee this won’t happen.

12

Turns out there’s a 2nd issue with this solution: if you do use some symbols from the library but not others you get a duplicate symbols error at link time because the first link retains the symbols you do use and the 2nd, force-link then re-defines them.

13

So, using the following approach should work because, by using target_link_options, force_load will be specified before the standard load of the library:

target_link_options(utils_video_encode_decode_ffmpeg PUBLIC "LINKER:-force_load,$<TARGET_FILE:avfilter>")
target_link_libraries (utils_video_encode_decode_ffmpeg PUBLIC avfilter)

I take this opportunity to show you a more portable approach for load option: use of LINKER: prefix and $<TARGET_FILE:> for library name.

1 Like
14

Thanks for the suggestion @marc.chevrier. Unfortunately that causes duplicate symbol errors which makes sense: the library is first force-linked and then we try to link it again. And we can’t get rid of the 2nd set of link flags because otherwise cmake will not ensure proper ordering and transitive dependencies.

15

I don’t think you will have problem with duplicate symbols if you have just a force_load and a regular load of the same static library if and only if the force_load occurs before the regular load. This is why I strongly recommend using the link options to specify the force_load because link options are specified before the libraries on the final link command line.

Now I reworked your example given on GitHub and I get all the force_load before regular libraries:

cmake_minimum_required(VERSION 3.7)
project(cpp C CXX)

# 4 imported libraries. Note that there are dependencies between them.
add_library(fake_1 STATIC IMPORTED GLOBAL)
set_target_properties(fake_1
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/imported_libs/libfake_1.a
)
add_library(fake_2 STATIC IMPORTED GLOBAL)
set_target_properties(fake_2
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/imported_libs/libfake_2.a)
target_link_libraries (fake_2 INTERFACE fake_1)

add_library(fake_3 STATIC IMPORTED GLOBAL)
set_target_properties(fake_3
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/imported_libs/libfake_3.a)
target_link_libraries (fake_3 INTERFACE fake_2)

add_library(fake_4 STATIC IMPORTED GLOBAL)
set_target_properties(fake_4
PROPERTIES
IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/imported_libs/libfake_4.a)
target_link_libraries(fake_4 INTERFACE fake_3)

add_subdirectory(lib_a)
add_subdirectory(lib_b)
add_subdirectory(exe)

lib_a subdir:


add_library(lib_a STATIC lib_a.cpp lib_a.h)
target_include_directories(lib_a PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

target_link_options(lib_a PUBLIC "LINKER:-force_load,$<TARGET_FILE:fake_4>"
                                 "LINKER:-force_load,$<TARGET_FILE:fake_3>")

lib_b subdir:


add_library(lib_b STATIC lib_b.cpp lib_b.h)
target_include_directories(lib_b PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
target_link_libraries(lib_b PUBLIC lib_a)
target_link_options(lib_b PRIVATE "LINKER:-force_load,$<TARGET_FILE:fake_2>")

exe subdir:


add_executable(exe ./main.cpp)
target_link_libraries(exe PRIVATE lib_b)

target_link_options(exe PUBLIC "LINKER:-force_load,$<TARGET_FILE:fake_1>"
                               "LINKER:-force_load,$<TARGET_FILE:fake_2>")

Final link command for exe:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++   -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Xlinker -force_load -Xlinker /cmake-example-master/imported_libs/libfake_1.a -Xlinker -force_load -Xlinker /cmake-example-master/imported_libs/libfake_2.a -Xlinker -force_load -Xlinker /cmake-example-master/imported_libs/libfake_4.a -Xlinker -force_load -Xlinker /cmake-example-master/imported_libs/libfake_3.a CMakeFiles/exe.dir/main.cpp.o  -o exe ../lib_b/liblib_b.a ../lib_a/liblib_a.a

But what I recommend for maximum flexibility is to introduce two set of imported targets:

  1. regular libraries (this is what you have already done)
  2. forced load libraries

So, depending on of the requirements, you can select one or the other… Using same approach (i.e. command target_link_libraries) and you have a better control over the order of libraries.

add_library (force_fake_1 INTERFACE)
target_link_options(force_fake_1 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_1>")

add_library(force_fake_2 INTERFACE)
target_link_options(force_fake_2 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_2>")
target_link_libraries(force_fake_2 INTERFACE force_fake_1)

add_library(force_fake_3 INTERFACE)
target_link_options(force_fake_3 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_3>")
target_link_libraries(force_fake_3 INTERFACE force_fake_2)

add_library(force_fake_4 INTERFACE)
target_link_options(force_fake_4 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_4>")
target_link_libraries(force_fake_4 INTERFACE force_fake_3)

So, for example, to build the lib_a we have:

add_library(lib_a STATIC lib_a.cpp lib_a.h)
target_include_directories(lib_a PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

target_link_libraries(lib_a PUBLIC force_fake_4)
16

Thanks for the suggestion @marc.chevrier. Unfortunately it doesn’t work for a few reasons:

  1. You removed the INTERFACE_LINK_LIBRARIES property from the IMPORTED libs thus removing the ordering requirements which made this an issue in the first place.
  2. Your final link command has only the force-linked libs, not the additional non-forced once which do respect the ordering constraints.
  3. I’ve already tried this (see notes in the github repo) and if you do both the force-link (via target_link_options so it comes first) and “regular” linking so CMake gets the lib depencies included and things are linked in the proper order then you do get duplicate symbol exceptions. You can’t see that in my simple github repo because the imported libs aren’t real but in my real project it happens. You could pick any 3 real static libraries and replace the fakes in the github repo and you’ll see the errors.

Thanks for all the help though!

17

comments:

  1. I replace INTERFACE_LINK_LIBRARIES by command target_link_libraries which have exactly same effect.
  2. Feel free to add regular libraries to complete the example.
  3. My proposition is completely different to what you describe in GitHub. Please read carefully my latest proposition including INTERFACE libraries for managing force_load link option.

Anyway, at minimum, provide an operational example. Currently, it is not possible to continue investigation without it.

18

I think I see what you were going for now @marc.chevrier. You have:

add_library(lib_b STATIC lib_b.cpp lib_b.h)
target_include_directories(lib_b PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
target_link_libraries(lib_b PUBLIC lib_a)
target_link_options(lib_b PRIVATE "LINKER:-force_load,$<TARGET_FILE:fake_2>")

which will cause lib_b to static link against fake_2. However, since you used "LINKER:-force_load,$<TARGET_FILE:fake_2"> cmake will know recognize that as fake_2 so it will not take into account the fact that fake_2 depends on fake_1 so I think the link order isn’t guaranteed (this was the original issue - once in quotes and with link flags it’s an opaque thing to cmake). To fix you’d need to add a target_link_libarary(lib_b PUBLIC fake_2) which would cause us to get fake_2's transitive dependencies. Unfortunately, it also causes the duplicate symbol error.

At the moment I’m experimenting with Bazel to see if it can fix our issues. I may come back to this depending on how those experiments go.

19

This is why I suggest the second solution, using separate definitions for regular and force libs.

With the second solution we have:

add_library(lib_b STATIC lib_b.cpp lib_b.h)
target_include_directories(lib_b PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
target_link_libraries(lib_b PUBLIC lib_a force_fake_2)

And because INTERFACE library force_fake_2 depends on INTERFACE library force_fake_1, you got the expected behavior.

20

Given

add_library (force_fake_1 INTERFACE)
target_link_options(force_fake_1 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_1>")

add_library(force_fake_2 INTERFACE)
target_link_options(force_fake_2 INTERFACE "LINKER:-force_load,$<TARGET_FILE:fake_2>")
target_link_libraries(force_fake_2 INTERFACE force_fake_1)

we’ve told cmake that force_fake_2 depends on interface library force_fake_1. However, since force_fake_1 is an interface library that doesn’t add anything to the link line directly. We have told cmake that if you use force_fake_1 you much put -force_load,$<TARGET_FILE:fake_>on the link line *somewhere* and it has to put-force_load,$<TARGET_FILE:fake_2>` somewhere but it’s not obvious to me that cmake knows that the order of those two link options matters. Are you sure we have guarantees there?