After you install CMake 3.28.1, could you then try to manually uninstall it before trying to install a newer CMake version?
I’m wondering here if there’s a stray registry value that is being left set somehow. Not sure I can offer much guidance on what that might be, it may take trawling through CMake’s packaging scripts to work that out. The relevant parts are likely to be in the files here.