systemd best practices

systemd best practices

Recommended patterns for writing reliable systemd unit files, securing services, and managing logs.

Use Drop-In Overrides Instead of Editing Vendor Unit Files

systemd loads override files from /etc/systemd/system/{service}.d/override.conf on top of vendor-provided unit files. Editing /usr/lib/systemd/system/ files directly causes package updates to overwrite changes. Use systemctl edit {service} to create drop-in overrides.

Run Services as Non-Root Users in systemd

systemd service unit files should specify User= and Group= to run processes with the minimum required privileges. Running services as root exposes the system to greater damage if the service is compromised. Create a dedicated system user for each application:

sudo useradd --system --no-create-home --shell /usr/sbin/nologin myapp

Set Restart Policies for Production systemd Services

Production services should use Restart=on-failure and a RestartSec delay to recover from transient failures without overwhelming the system. Restart=always is appropriate for critical services that must never be down. Avoid Restart=no (the default) for production workloads.

Enable Persistent Journal Storage for systemd

systemd stores journal data in volatile memory ( /run/log/journal/) by default on some distributions, losing logs on reboot. Create /var/log/journal/ and set Storage=persistent in /etc/systemd/journald.conf to retain logs across reboots.

Use Hardening Directives in systemd Unit Files

systemd provides security sandboxing directives that restrict what a service can access. Add these to the [Service] section of unit files for defense in depth:

[Service]
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
NoNewPrivileges=true
ReadWritePaths=/var/lib/myapp

ProtectSystem=strict makes the entire filesystem read-only except paths listed in ReadWritePaths=. NoNewPrivileges=true prevents the service from gaining elevated permissions through setuid binaries.

Limit Journal Size to Prevent Disk Exhaustion

systemd journal files can grow unbounded and fill the disk. Set SystemMaxUse= in /etc/systemd/journald.conf to cap total journal size:

[Journal]
SystemMaxUse=500M
SystemMaxFileSize=50M