I dit understand you the whole time, but you expect that CMake is doing the work for you!
It is an usage error, not a CMake error
bash-5.2$ git diff *.in
diff --git a/lib/mylibrary.pc.in b/lib/mylibrary.pc.in
index d073438..e7fa2e2 100644
--- a/lib/mylibrary.pc.in
+++ b/lib/mylibrary.pc.in
@@ -1,8 +1,11 @@
-libdir=@CMAKE_INSTALL_FULL_LIBDIR@
-includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
+# libdir=@CMAKE_INSTALL_FULL_LIBDIR@
+# includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
+libdir=${pcfiledir}/../@CMAKE_INSTALL_LIBDIR@
+includedir=${pcfiledir}/../../@CMAKE_INSTALL_INCLUDEDIR@
Name: @PROJECT_NAME@
Description: PNG reader
-Version: @VERSION@
-Libs: -L${libdir} -lmylibrary @MYLIBRARY_LINK_LIBRARIES@
-Cflags: -I${includedir} @MYLIBRARY_INCLUDE_DIRECTORIES@
+Version: @PROJECT_VERSION@
+Requires: libpng >= 1.6
+Libs: -L${libdir} -lmylibrary -lpng16 # XXX @MYLIBRARY_LINK_LIBRARIES@
+Cflags: -I${includedir} -I@PNG_INCLUDE_DIRECTORIES@ # -I/usr/local/opt/libpng/include/libpng16 # XXX @MYLIBRARY_INCLUDE_DIRECTORIES@
This works fine 
see too https://bugs.freedesktop.org/show_bug.cgi?id=62018
and How to generate .pc (pkg-config) file supporting --prefix of the cmake --install? - #6 by craig.scott