You don’t get to “ignore” --install --prefix at the project level, it has nothing to do with the project really. It is a tool for packagers, the downstream users who are building the project for whatever purpose.
The project is only aware of the configure-time prefix, CMAKE_INSTALL_PREFIX, it has no ability to interact with a prefix given at install time.
The purpose of the option is simple, it allows the install to occur to an arbitrary location. This is necessary for most packaging workflows. Imagine you are producing a tarball of the installed project. You must run the project’s install command, then collect the files into the archive.
If you installed the project directly into the system’s root folder, /usr/bin, /usr/include, etc. it would be very difficult to collect only the files relevant to the project from those locations. The solution is to install the project to some arbitrary, temporary prefix, with --install --prefix, then archive the files which appear there.
Well behaved projects generally shouldn’t care about the install prefix at all, at configuration or install time. Paths should be made relative within the install tree, allowing the overall tree to be relocatable so long as its structure is preserved.