Setting compiler flags in CMakeLists.txt

It seems like CMake uses CMAKE_CXX_FLAGS_RELEASE and CMAKE_CXX_FLAGS_DEBUG (depending on the current value of CMAKE_BUILD_TYPE) in addition to CMAKE_CXX_FLAGS. The -O3 you see is likely set by default in CMAKE_CXX_FLAGS_RELEASE.

If you want to use only the flags you manually specified, you can set the build type-specific flags to the empty string like this:

SET( CMAKE_CXX_FLAGS_DEBUG  "")
SET( CMAKE_CXX_FLAGS_RELEASE  "")
SET( CMAKE_CXX_FLAGS  "-Ofast -DNDEBUG -std=c++20 -march=native -fpic -ftree-vectorize")