I don’t know what “cmake clean” is, I assume you meant a typical “make clean” or its equivalent for your chosen build tool.
Take a look at the ADDITIONAL_CLEAN_FILES directory property. Despite its name, it does allow you to list directories too (says so in its docs).