I have never use precompiled header and I would not do it.
How will you install only our public interface header only?
Use:
target_sources(targetName
<PRIVATE|PUBLIC|INTERFACE>
FILE_SET setName
[TYPE fileType]
[BASE_DIRS dir1 [dir2...]]
[FILES file1 [file2...]]
... )
Than your project will be much simpler and clear.
But perhaps this Include directories for target_precompile_headers may help you.