OSX, frameworks, @rpath, and image not found

Hi,
I wonder how @rpath is supposed to work on OSX/BigSur …

I build a framework with CMake, that basically works
But all tests are failing with a message basically saying the framework can not be found, but it’s there.

./build/bin/my_test 
dyld: Library not loaded: @rpath/MyProject.framework/Versions/0.0.0/MyProject
  Referenced from: /Users/a4z/project/./build/bin/my_test
  Reason: image not found
zsh: abort      ./build/bin/my_test


-> ls build/lib/MyProject.framework/Versions/0.0.0/MyProject 
build/lib/MyProject.framework/Versions/0.0.0/MyProject

I tried to set , as described here, https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling, but have no success.

Any idea what I could do to run the tests using the framework ?

PS: creating a dynlib, not a framework, works when I set the LD_LIBRARY_PATH , but since we build also for iOS, I need a framework …

PPS: using install_name_tool to replace the rpath with an absolute path seem to work …

macOS rpath handling is still kind of wonky with CMake. CMake doesn’t carry “rpath usage requirements” as part of its interface requirements, so it doesn’t know that it needs to add an LC_RPATH entry to consuming libraries. At least this is certainly true for external libraries and frameworks. Those in the same project should work though, but maybe that’s because I tend to handle rpath settings at the very top rather than at specific locations within the tree.

Can you share the link line for my_test?

In reality, it is of course not my test lib …
remove one test binary and run a verbose build, that is the output

 cmake --build build/Mac/nativeNinja -v          
[0/2] /usr/local/Cellar/cmake/3.19.7/bin/cmake -P /Users/a4z/elux/nsdk/ecp-sdk-native/build/Mac/nativeNinja/CMakeFiles/VerifyGlobs.cmake
[1/1] : && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -m64 -stdlib=libc++  -m64 -stdlib=libc++ -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -m64 src/ecp-sdk-native/tests/CMakeFiles/_ecp_sdk_test.dir/sdkversion.test.cpp.o -o bin/_ecp_sdk_test  lib/EcpSdkNative.framework/Versions/0.0.0/EcpSdkNative  lib/libunittests.a  lib/libdjinni.a  /Users/a4z/elux/nsdk/.conan/data/miniz/2.1.0/_/_/package/0e07c2e87d8cbf9a0ea738ec47d91d4f8edb54d2/lib/libminiz.a  lib/lib_appliance.a  lib/lib_builder.a  lib/lib_datastore.a  /Users/a4z/elux/nsdk/.conan/data/lua/5.3.5/_/_/package/111303eadd93c973f5fff154c74a4dc8d15fd30a/lib/liblua-c++.a  lib/lib_session.a  /Users/a4z/elux/nsdk/.conan/data/paho-mqtt-c/1.3.8/_/_/package/2af9799d495ec2a1473b323d61820626ca0e5d74/lib/libpaho-mqtt3as.a  -lc  -lpthread  lib/lib_host.a  lib/libdjinni_statemachine.a  /Users/a4z/elux/nsdk/.conan/data/djinni-support-lib/0.0.1/_/_/package/736c4c96d5832ddaa0b8546c39ad86b81b0c4880/lib/libdjinni_support_lib.a  lib/libwifi.a  -framework Foundation  -framework CoreWLAN  lib/libelux-alljoyn.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/libajrouter.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_about.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_config.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_controlpanel.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_notification.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_onboarding.a  /Users/a4z/elux/nsdk/.conan/data/alljoyn/0.16.10/_/_/package/6de3cba6e25c793402f50d68e53eb72b761e0f7d/lib/liballjoyn_services_common.a  lib/lib_jobs.a  lib/lib_ecpcalls.a  lib/lib_httpsclient.a  /Users/a4z/elux/nsdk/.conan/data/libcurl/7.75.0/_/_/package/2f049a9b3c26e8134014910d7a0513812511d459/lib/libcurl.a  /Users/a4z/elux/nsdk/.conan/data/openssl/1.1.1j/_/_/package/0e07c2e87d8cbf9a0ea738ec47d91d4f8edb54d2/lib/libssl.a  /Users/a4z/elux/nsdk/.conan/data/openssl/1.1.1j/_/_/package/0e07c2e87d8cbf9a0ea738ec47d91d4f8edb54d2/lib/libcrypto.a  /Users/a4z/elux/nsdk/.conan/data/zlib/1.2.11/_/_/package/0e07c2e87d8cbf9a0ea738ec47d91d4f8edb54d2/lib/libz.a  lib/lib_coreutils.a  /Users/a4z/elux/nsdk/.conan/data/fmt/7.1.3/_/_/package/858013db89e03a2cd825b12ab854c4e026401118/lib/libfmt.a && :

the && : at the end is interesting
the framework in the same apackage is lib/EcpSdkNative.framework/Versions/0.0.0/EcpSdkNative

There no rpath flag in that link line. The && : is the placeholder for any POST_LINK custom commands. Does it work if you set CMAKE_BUILD_RPATH (the variable) to a list of paths you need to use the framework in question before the target(s) that need it?

interestingly, adding set(CMAKE_BUILD_RPATH "${CMAKE_CURRENT_BINARY_DIR}/lib") to the top of the top cmake file does not change anything, still no rpath flag in the command line for the linker

Have you tried the latest 3.20.1 release (just came out today)? There’s a fix in that which addresses an issue that had been preventing rpath support for iOS.

did update cmake now (brew) and have 3.20.1, did not change.
The behaviour is also not for iOS but for OSX/BigSur

I think I will have to utilise CMAKE_INSTALL_NAME_TOOL and create a smaller example that I can share as whole, maybe with a build on OSX via github actions or something similar. But that will not happen too soon …

For that very project the whole cmake it too big and not structured as I’d like it to be …

I think distilling the case into a simple reproducer would be useful here. Either the CMake issue is easier for us to resolve with the case or the missing piece is found in trying to making it. CMake can definitely add rpath entries to libraries, but something else seems to be interfering here.

Here’s a rough outline of a project that creates an app bundle with frameworks:

set(CMAKE_BUILD_WITH_INSTALL_RPATH YES)

add_executable(MyApp MACOSX_BUNDLE ...)
add_library(Fmwk1 SHARED ...)
add_library(Fmwk2 SHARED ...)

# Dependencies: MyApp --> Fmwk1 --> Fmwk2
# Direct linking like this assumes CMake 3.19 or later
target_link_libraries(MyApp PRIVATE Fmwk1)
target_link_libraries(Fmwk1 PRIVATE Fmwk2)

# Not all required properties shown below, just those related to these discussions
set_target_properties(MyApp PROPERTIES
    INSTALL_RPATH @executable_path/../Frameworks  # Assumes macOS bundle layout
    # CMake 3.20 or later required for the following
    XCODE_EMBED_FRAMEWORKS Fmwk1 Fmwk2
    XCODE_EMBED_FRAMEWORKS_CODE_SIGN_ON_COPY TRUE
    XCODE_EMBED_FRAMEWORKS_REMOVE_HEADERS_ON_COPY TRUE
)
set_target_properties(Fmwk1 Fmwk2 PROPERTIES
    FRAMEWORK TRUE
    INSTALL_RPATH @loader_path/../../..  # Assumes macOS bundle layout
    XCODE_ATTRIBUTE_CODE_SIGNING_ALLOWED FALSE
)

Some key observations:

  • BUILD_WITH_INSTALL_RPATH is usually what you want in the Apple world. An app bundle should be laid out with exactly the same directory structure whether in the build tree or whether installed (and workflows using the Xcode generator are quite likely to have no install step). You should be able to use the same relative @rpath locations even in your build tree. You don’t want/need absolute paths embedded in the binaries if you’ve got things set up correctly.
  • The above linking is taking advantage of improvements added in CMake 3.19. With earlier versions, linking frameworks presents problems when it comes time to embed them in the app bundle. CMake 3.20 completed that work and now directly supports framework embedding when using the Xcode generator. I’ve shown key steps for that in the above as well.

The above is for an app bundle, which your test appears to not be. I’ve shown the above mostly because it highlights how a typical arrangement may present complexities for test executables because those executables lack an app bundle structure. That would mean frameworks won’t be found at the places the embedded paths indicate. The test target should probably not set BUILD_WITH_INSTALL_RPATH to true because the test won’t be installed. You specifically want to embed absolute paths there. Perhaps in your project you’ve set the CMAKE_BUILD_WITH_INSTALL_RPATH variable like in my example and it is being applied to your test targets? If you prevent that, does it give you a working test? You would probably have to make the test executable link directly to all frameworks it needs, even if it doesn’t directly refer to any symbols from some of those frameworks (the linker may defeat you here if it optimises out some of those linker dependencies in the test binary).

You may want to look at XCTest. While I don’t have direct experience with it myself, it does appear to be potentially related to your situation (testing frameworks). CMake has a FindXCTest module which may help setting that up. You may want CMake 3.19.5 or later for that due to this issue.

Thanks a lot @craig.scott !
Sorry for the late reply , busy times.

I re-ordered some some things int the cmake and now it works.
Excecpt, I have now a small other project that has an enable_language(OBJC)
It does C++, Objective-C and Swift, and the objective-C part needs a ${CMAKE_INSTALL_NAME_TOOL} -add_rpath "@executable_path/../lib"

I will be able to point at the example project very soon.

Ad XCTest, I think Apple broke that today
See: Apple Developer Documentation

Deprecations
Xcode no longer includes XCTest’s legacy Swift overlay library (libswiftXCTest.dylib). 
Use the library’s replacement, libXCTestSwiftSupport.dylib, instead. 
For frameworks that encounter build issues because of the removal of the legacy library, delete the framework and library search paths for libswiftXCTest.dylib and add the ENABLE_TESTING_SEARCH_PATHS=YES build setting.
This setting automatically sets the target’s search paths so that it can locate the replacement XCTest library. (70365050)

Yesterday my Swift tests where building fine, today I get : error: cannot find 'XCTAssert' in scope messages …after the XCode update, and I guess my new problems have something to do with this