If you’re using the URL method and don’t specify a hash, I believe the default behavior is that it’s redownloaded every time. I typically use the external project option
UPDATE_DISCONNECTED true
Or for all ExternalProject in that directory scope
set_property(DIRECTORY PROPERTY EP_UPDATE_DISCONNECTED true)
That should stop the constant redownloading on every configure.
I also like external project option
CONFIGURE_HANDLED_BY_BUILD true
That in general will save significant time of reconfiguring external projects besides just the redownloading.