You should give us more informations on how it fails. From my personal experience if parallel build fails (sometimes but not always) and sequential build never fails you probably miss some dependencies in you project so that parallel build cross the unspecified dep more often, because it start at several places in the dep tree.
Which version of CMake are you using?
Which generator do you usually use?