Don’t confuse the PUBLIC keyword with meaning “part of the public API that consuming targets will want to include”. Those are two very different things. The PUBLIC keyword in target_sources() is about which targets those sources should be added to directly, not which headers should consumers be allowed to #include.
As I stated, I can’t think of a scenario where I would use PUBLIC for target_sources(). There might be one, but I am not aware of it. You would have to look at what the project is trying to do, but if it uses PUBLIC with target_sources(), it is very likely to be incorrect.
If your question is whether adding headers as INTERFACE with target_sources() is valid, I touched on that in the first dot point of my previous comment. It has been used for a long time with header-only libraries (which are relatively common). It’s not wrong to do this, but it might not be what everyone would want.