CMake config-only memory usage

Usage
1

I am using cmake on libwebsockets for a long time, with great success.

Recently I have been working on our CI, in particular splitting the build process into separate spawn actions each of which records the cpu usage and peak RSS allocations, with the eventual goal of autolearning how to pack the different build steps on a particular builder for best efficiency. libwebsockets offers many build options, so we build it many different ways in CI to see if we broke some particular aspect.

This has caused me to notice that the memory usage behaviour of cmake (just the configuration part, not the make / ninja part, which is done separately) on the same project with different options, is VERY different. For almost all options, cmake’s peak RSS usage is close to 300MB, which is fine when considering the smallest builder has 4GiB RAM. On one build variant though, ( cmake .. -DLWS_WITH_DISTRO_RECOMMENDED=1 -DLWS_WITH_MINIMAL_EXAMPLES=1 ), the RSS usage on cmake balloons to 3.8GiB.

■ >saib> Step 3: [ 1:15min ( 51.671s u / 21.310s s), Mem: 3.820GiB, Stg: 96.884MiB ]

This is still OK for builders that have more RAM, from which I pasted the above, but on two builders, they can’t cope and the OOM killer steps in.

LWS_WITH_MINIMAL_EXAMPLES option is the default now and is used in most of the variants with the cmake memory usage still at 300MiB. LWS_WITH_DISTRO_RECOMMENDED is a compound option that enables many other options, with the idea distros can use it to one-shot the behaviours they are likely to want from a single lws build (support all the different event loop libs etc).

I will pick this compound option apart to try to isolate what causes the increased memory usage, but I am wondering if this is an expected behaviour under some conditions? I used the latest 4.1.1 on the kitware apt repos (this broke on an Ubuntu builder) but it didn’t change the behaviour.

2

This was quite confusing, when reproducing on the same box, with the same commit checked out and the same env and same args to cmake, I could “only” get it to consume 1.6GiB peak, which is workable (although let’s face it, bloated).

I figured out eventually the “cause” is the base dir length, if I add about 70 bytes to the cwd this is built from, so from /home/agreen/libwebsockets/build to /home/agreen/q/4645496C370B952279771D000E700B10695CF3E2E0F8ADDCD900FC193489392/libwebsockets/build I can reproduce it bloating to more than the physical memory on the box and OOM killing it; the cwd is then aligned to what the CI cwd looks like.

So at least the thing that tips it over the edge is that cmake seems to be willing to consume GiBs of RAM repeating the base dir path over and over. This is surely something that could be represented in cmake more cheaply?

3

This turned out to be my own fault, in the CMakeLists.txt when building subdirectories, it was accidentally repeatedly compounding the env var for include directories until it was a monsterous jumble (a semicolon-separated list of 18K members being copied around).

After it was fixed in lws, peak memory reduced to ~93MB from>2GiB or double that if the cwd was long. Since it was not “wrong”, just containing the same paths over and over it did not otherwise create any symptom except the memory usage and time eaten for nothing.