Service Management with systemd

Updated

September 12, 2026

Service Management with systemd

Hand-written unit files in /etc/systemd/system/ plus daemon-reload is how two hosts disagree. The boring default is: systemd.services.<name> as a Nix attrset, store-path binaries, and a high-level services.<app>.enable module when nixpkgs already has one.

Mental model

NixOS renders unit files into the system closure. nixos-rebuild switch diffs old vs new units and restarts what changed. You do not systemctl enable by hand; wantedBy = [ "multi-user.target" ]; is enablement.

Two layers:

Layer When
services.nginx.enable = true; nixpkgs already knows nginx. Use it.
systemd.services.desk-worker = { … }; The desk binary has no module yet.

Custom units need absolute ExecStart. The service PATH is not your login PATH. ${pkgs.coreutils}/bin/echo, not echo.

Hardening that costs one line:

Setting Effect
DynamicUser = true; Ephemeral uid; no useradd
ProtectSystem = "strict"; Root filesystem read-only
ProtectHome = true; /home inaccessible
PrivateTmp = true; Private /tmp
NoNewPrivileges = true; No privilege escalation 
systemd.services.desk-api
   wantedBy  →  starts at boot
   after     →  ordering
   ExecStart →  store path
   Restart   →  on-failure

Worked examples

Case 1: A desk worker unit

Save as desk_service_unit.nix:

# desk_service_unit.nix
{ pkgs, ... }:

{
  systemd.services.desk-worker = {
    description = "Desk operations worker";
    wantedBy = [ "multi-user.target" ];
    after = [ "network-online.target" ];
    wants = [ "network-online.target" ];
    # network.target is "stack is up", not "have a default route".
    # Bind-to-0.0.0.0 workers can use network.target. Outbound HTTP needs online.

    environment = {
      DESK_ENVIRONMENT = "production";
      PORT = "8080";
    };

    serviceConfig = {
      Type = "simple";
      ExecStart = "${pkgs.writeShellScript "desk-worker" ''
        ${pkgs.coreutils}/bin/echo "Desk worker on port $PORT"
        while true; do ${pkgs.coreutils}/bin/sleep 3600; done
      ''}";
      Restart = "on-failure";
      RestartSec = "5s";
      DynamicUser = true;
      ProtectSystem = "strict";
      ProtectHome = true;
      NoNewPrivileges = true;
      PrivateTmp = true;
      MemoryMax = "256M";
    };
  };
}
sudo nixos-rebuild switch
systemctl status desk-worker
journalctl -u desk-worker -n 20 --no-pager

Output (shape):

● desk-worker.service - Desk operations worker
     Loaded: loaded (/etc/systemd/system/desk-worker.service; enabled)
     Active: active (running)

/etc/systemd/system/desk-worker.service is a symlink into /nix/store. Editing it by hand is undone on the next switch.

Case 2: Prefer a nixpkgs module when it exists

Save as nginx.nix:

# nginx.nix
{ pkgs, ... }:

{
  services.nginx = {
    enable = true;
    virtualHosts."desk.internal" = {
      root = pkgs.writeTextDir "index.html" "<h1>Desk</h1>";
    };
  };
}

This opens the right units, user, and (optionally) firewall helpers. Do not copy a 40-line systemd.services.nginx from a wiki.

systemctl is-active nginx
curl -sS http://127.0.0.1/ | head

Case 3: Timers instead of cron

Save as desk_timer.nix:

# desk_timer.nix
{ pkgs, ... }:

{
  systemd.services.desk-cleanup = {
    description = "Clean stale desk tickets";
    serviceConfig = {
      Type = "oneshot";
      ExecStart = "${pkgs.coreutils}/bin/echo desk-cleanup-ok";
    };
  };

  systemd.timers.desk-cleanup = {
    wantedBy = [ "timers.target" ];
    timerConfig = {
      OnCalendar = "daily";
      Persistent = true;
    };
  };
}
systemctl list-timers --all | grep desk
sudo systemctl start desk-cleanup.service
journalctl -u desk-cleanup -n 5 --no-pager

Persistent = true catches up after the VM slept through midnight.

Case 4: Secrets as files, not unit text

# desk-envfile.nix
{
  systemd.services.desk-worker.serviceConfig.EnvironmentFile =
    "/run/secrets/desk-worker.env";
}

environment.PORT = "8080"; is fine for non-secrets. A database password in environment.PASSWORD = "…" lands in the world-readable store. EnvironmentFile points at a sops-decrypted tmpfs path (secrets part). The unit file then contains a path, not the secret.

LoadCredential = "db:${config.sops.secrets.db.path}"; is the systemd-native sibling: the credential appears under $CREDENTIALS_DIRECTORY/db with mode 0400 for the service uid. Prefer it over leaking Environment= when the app can read a file. Type = "notify" only if the binary calls sd_notify; a shell while sleep is simple. A oneshot timer job is Type = "oneshot" (Case 3) — Restart = "always" on a oneshot is a fork bomb.

Case 5: See what switch will restart

sudo nixos-rebuild dry-activate --flake .#desk-workstation

Output (shape):

would restart the following units: desk-worker.service
would not restart: sshd.service

If dry-activate says it will restart sshd because you changed a comment in a shared module, split the module. Restarts are part of the change.

The trap

The trap is ExecStart = "desk-api" or ExecStart = "/usr/bin/curl …"**. The unit’s PATH is minimal. desk-api: command not found at boot, while your interactive shell still has it.

Always ${pkgs.desk-api}/bin/desk-api or a writeShellScript that uses store paths inside.

The other trap is systemctl edit desk-worker drop-ins. They live outside the flake. The next switch may fight them, or they survive and surprise the next operator. If you needed the drop-in, it belonged in systemd.services.desk-worker.serviceConfig.

The boring rule

  • High-level services.<app> when nixpkgs has it. Raw systemd.services for desk-specific units.
  • ExecStart is a store path. No bare names, no /usr/bin.
  • DynamicUser + ProtectSystem = "strict" on network daemons that do not need a named uid.
  • Timers, not crontab.
  • Secrets via EnvironmentFile / LoadCredential / sops paths, never environment.SECRET.
  • network-online.target only when you need a route. Type=notify only when the app notifies.

Try this

  1. Add serviceConfig.MemoryMax = "64M"; to the worker, switch, systemctl show desk-worker -p MemoryMax.
  2. Change ExecStart to echo hello (no store path), switch, journalctl -u desk-worker -n 20. Restore the store path.
  3. systemctl cat desk-worker and confirm the unit lives under /nix/store.
  4. Enable services.timesyncd (or services.chrony) via the module, not a custom unit. systemctl status systemd-timesyncd.