There are some less mainstream platforms where you can actually link to an executable. For those, PUBLIC can make sense. But in practice, unless that specific platform is what you’re targeting and you specifically want to take advantage of that pattern, I always recommend using PRIVATE. That communicates the generally intended relationship.
I also recommend always specifying the visibility (PRIVATE, INTERFACE, or PUBLIC). Much older CMake versions didn’t support the visibility keywords, but that was many years ago now. I sometimes think we should consider making the visibility keyword required, but that would be a noisy policy change for older projects.