Limit a group of services (eg apache, mysql, php-fpm) to restrict Memory and CPU with systemd and cgroups v2

A long title, for what is actually a simple thing, which nobody seems to have written a simple howto for. 

Here we go.

Let’s say you have Apache, Mysql and PHP-FPM which you want to limit, as a group, the memory and CPU usage of, so as to allow some “headroom” if things hit the fan.

In the old days you would use cgconfig.conf, cgrules daemon. cgconfigparser…

In the new days your Linux operating system is probably systemd based, and this is how you do it (tested Ubuntu 22.04).

There are two “oom killers” in play here, the kernel itself, which should be seen as a “last resort”, it might be too slow to save a system, and the systemd-oomd proactive killer, which should step in before things get that bad. 

First, create a “slice”, a “slice” is just a fancy systemd name for a cgroupsv2 group, create a file /etc/systemd/limited.slice with following contents

[Unit]

Description=Apache MySQL and PHP-FPM Slice

Documentation=man:systemd.special(7)

Before=slices.target

execute the folowing to configure the limits for this slice (make up your own limits as appropriate), these will persist through reboots…

sudo systemctl set-property limited.slice MemoryHigh=128M     # This is the “soft” limit of actual ram

sudo systemctl set-property limited.slice MemoryMax=256M      # This is the “hard” limit of actual ram

sudo systemctl set-property limited.slice MemorySwapMax=512M  # This is the “hard” limit of swap

sudo systemctl set-property limited.slice CPUQuota=95% 

execute the following to put the services into the slice, again, this will persist through reboots…

     cd /etc/systemd/system.control 

for srv in apache2 mysql php5.6-fpm php7.3-fpm php8.0-fpm    # adjust as required 

do

    sudo mkdir $srv.service.d

    echo "[Service]" | sudo tee -a ${srv}.service.d/10-Slice.conf >/dev/null

    echo "Slice=limited.slice" | sudo tee -a ${srv}.service.d/10-Slice.conf >/dev/null

    sudo systemctl daemon-reload

    sudo service  ${srv} restart

done

That is enough for the kernel oom killer to work, but we should also add the userspace one,

    sudo apt install systemd-oomd
    sudo systemctl set-property limited.slice  ManagedOOMSwap=kill 
    sudo systemctl set-property limited.slice  ManagedOOMMemoryPressure=kill 
    sudo systemctl set-property limited.slice  ManagedOOMMemoryPressureLimit=90%
    sudo systemctl set-property limited.slice  ManagedOOMPreference=none

Note that the Swap limit is set in  /etc/systemd/oomd.conf (defaults to 90%)

If you want, for example, to avoid having mysql stopped (leaving apache and php-fpm instances to be stopped first)

    sudo systemctl set-property mysql.service  ManagedOOMPreference=avoid

And that should about do it, you have created a systemd slice (cgroup) with limited memory and cpu, and configured systemd to put those listed services into that slice.  These changes will persist through reboots so you only have to do this once.

You can check your work with  systemd-cgls which will show you the slices and that the processes are in said slices and oomctl which will show you which groups have the userspace oom enabled.

By default when the (userspace) oom killer kills a process the servcie will be “stop”ped, if you want the service to continue running, you can set that at the service level

    sudo systemctl set-property mysql.service  OOMPolicy=continue