Help with recompiling on a cluster

Hi all,

I use cmake on a cluster of systems where for the most part I don’t control what node I get when submitting a job.

I find that when I build my cmake project, it does a full recompile all the time instead of incremental. As far as I can tell,
it appears to happen because the job lands on a different host than the last time.

Is there a way in cmake to sidestep this behavior and allow it to treat different nodes as the same for configuration
purposes? All the nodes have the same software configuration and filesystems are all shared.

Thanks
Jerry

1 Like

I looked into it a little bit. Apologies I’m not familiar with compiling on clusters.

Perhaps this blog might interest you:

It talks about using distcc:

“distcc is a program to distribute compilation of C or C++ code across several machines on a network. distcc should always generate the same results as a local compile, is simple to install and use, and is often two or more times faster than a local compile.”

It sounds basically perfect for your use case and has integration with CMake via the usage of CMAKE_LANG_COMPILER_LAUNCHER

Hi and thanks for taking time to respond to my question.

It looks like distcc is a very static environment that needs servers set up for a compiler farm. I would have to launch these servers dynamically (and I don’t get to choose the machines the servers land on), check that they are up and running, then relaunch compilation that can use them. This would be quite messy and from my reading so far, distcc doesn’t seem to address this kind of more dynamic environment.

Perhaps I’m mistaken, but I don’t think it will work amazingly well.

In our env, it’s much simpler to launch a single job with N cores such as 40 to do the compile. As long as I don’t have to recompile the whole tree every time :slight_smile:

I would start by asking the build tool why it thinks it needs to do this. make -d and ninja -d explain for the “normal” tools.