You did not mention how your build command looks like but adding build tool specific options may speed up the build process a lot even for MSBuild.
By default MSBuild does not try to build in parallel. You may add add_compile_definitions(/MP) for MSVC to your CMakeLists.txt to enable that, but it can also be done using MSBuild command line arguments. For building our solutions with MSBuild in parallel we are using the following command
cmake --build builddir --target ALL_BUILD --config Release -- /nologo /verbosity:minimal /maxcpucount:2 /property:MultiProcessorCompilation=true /l:FileLogger,Microsoft.Build.Engine;logfile=logfilename.log
Hope that helps.