I haven’t tried your case to see if there’s a straightforward way with find_library. However, in general when I need to test something that is easiest to do from a small project, I make a subdirectory and put a simple CMakeLists.txt. I then test that project at CMake configure time using try_compile from the main project.
Example where the fs_check directory is used via try_compile to ensure that c++ standard library filesystem is linkable and actually working. Because the feature check macros don’t always ensure that the standard library works for that particular case.