Ephemeral Roots and Impermanence

Updated

September 12, 2026

Ephemeral Roots and Impermanence

/usr/local/bin leftovers, an undocumented /etc drop-in, a rootkit in /bin — all of that is “state you did not declare.” The boring default is: throw / away at boot (tmpfs or a blank Btrfs snapshot), keep /nix and /persist, and list every surviving path in the impermanence module.

This is optional on a laptop you are still learning on. It is the right default on a desk server once Disko already created @persist and @nix. Pair it with the backups chapter: git rebuilds the OS; restic rebuilds /persist.

Mental model

Path Survives reboot?
/ (tmpfs or wiped subvol) No
/nix Yes (own subvolume / partition)
/persist Yes
/boot Yes (ESP)
Anything not listed in environment.persistence No

The impermanence module bind-mounts or links /persist/… onto the live paths (/etc/machine-id, /var/log, …) so programs still write where they always did.

boot
  tmpfs → /
  btrfs @nix → /nix
  btrfs @persist → /persist
  bind  /persist/etc/machine-id → /etc/machine-id

If you forget a path, the service fails after reboot (empty DB, new SSH host key). That failure is the audit. Add the path, or decide the data should die.

Also persist what decrypts secrets: /etc/ssh/ssh_host_ed25519_key (sops sshKeyPaths) and/or /var/lib/sops-nix. A tmpfs root plus a new host key every boot is a machine that never decrypts secrets/desk.yaml.

Worked examples

Case 1: tmpfs root + persist list

Save as impermanence.nix:

# impermanence.nix
{ inputs, ... }:

{
  imports = [ inputs.impermanence.nixosModules.impermanence ];

  fileSystems."/" = {
    device = "none";
    fsType = "tmpfs";
    options = [ "defaults" "size=4G" "mode=755" ];
  };

  fileSystems."/persist" = {
    neededForBoot = true;
  };

  environment.persistence."/persist" = {
    hideMounts = true;
    directories = [
      "/var/log"
      "/var/lib/nixos"
      "/var/lib/systemd/coredump"
      "/var/lib/systemd/timers"
    ];
    files = [
      "/etc/machine-id"
    ];
  };
}

fileSystems."/persist" device/fsType usually come from Disko. Do not fight Disko with a second definition of the same mount unless you are not using Disko.

neededForBoot = true is required so activation can see /persist before it tries to link machine-id.

Case 2: SSH host keys — the one everyone forgets

# persist-ssh.nix
{
  environment.persistence."/persist".files = [
    "/etc/ssh/ssh_host_ed25519_key"
    "/etc/ssh/ssh_host_ed25519_key.pub"
  ];
}

Without this, every reboot is a new host key. Clients print REMOTE HOST IDENTIFICATION HAS CHANGED. That looks like MITM. It is you.

After first boot with persistence, ls -l /persist/etc/ssh/ should show the keys. If the files exist only under /etc/ssh on a still-persistent root, copy them to /persist before you enable tmpfs root, or the next boot generates new keys.

Case 3: Desk service state

# persist-desk.nix
{
  environment.persistence."/persist".directories = [
    "/var/lib/desk-api"
    "/var/lib/postgresql"
    "/var/lib/tailscale"
    "/var/lib/nixos"
  ];
}

If the unit uses DynamicUser, state may live under /var/lib/private/…. Persist that path or give the service a named user and a stable StateDirectory. Guessing wrong looks like “the database reset.”

Users’ homes:

{
  environment.persistence."/persist".users.deskadmin = {
    directories = [
      ".ssh"
      ".local/share"
    ];
    files = [
      ".bash_history"
    ];
  };
}

Do not persist all of /home “to be safe” if the point was to stop accumulating junk. Persist keys and the few tools that store tokens.

Case 4: Flake input

# flake.nix fragment
{
  inputs.impermanence.url = "github:nix-community/impermanence";

  outputs = { self, nixpkgs, impermanence, ... }: {
    nixosConfigurations.desk-server = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        impermanence.nixosModules.impermanence
        ./impermanence.nix
        ./configuration.nix
      ];
    };
  };
}
nix eval .#nixosConfigurations.desk-server.config.environment.persistence.\"/persist\".files

Output (shape):

[ "/etc/machine-id" "/etc/ssh/ssh_host_ed25519_key" … ]

If eval does not list the SSH keys, reboot will not either.

Case 5: Prove it on a VM

On a lab VM after persistence is on:

sudo touch /etc/untracked-state.txt
sudo touch /persist/i-should-survive.txt
sudo reboot

After boot:

ls /etc/untracked-state.txt
ls /persist/i-should-survive.txt
ls /etc/machine-id

Output:

ls: cannot access '/etc/untracked-state.txt': No such file or directory
/persist/i-should-survive.txt
/etc/machine-id

machine-id is back because it was listed. The untracked file is gone. If machine-id is also gone, neededForBoot or the files list is wrong — fix before you enable sshd clients.

Btrfs variant (no tmpfs): rollback @root to a blank snapshot in boot.initrd.postDeviceCommands (or the current impermanence btrfs helper). Same persist list. Prefer one mechanism; do not tmpfs and rollback.

The trap

The trap is enabling tmpfs root without copying SSH host keys onto /persist first. You lock yourself out over SSH. Console or a second VM disk is the recovery.

The other trap is forgetting /var/lib/nixos (uids for DynamicUser / allocated ids). After reboot, users get new uids and cannot read old state you did persist.

A third trap: persisting /etc wholesale. You are back to a mutable /etc with extra steps. List files.

The boring rule

  • /nix and /persist (and /boot) are durable. / is not.
  • List SSH host keys, machine-id, /var/lib/nixos, and each service’s state dir before the first ephemeral boot.
  • neededForBoot = true on /persist.
  • Copy existing keys to /persist before switching root to tmpfs.
  • Back up /persist. The flake will not grow the Postgres data back.

Try this

  1. Eval the persist files and directories lists from the flake. Read them aloud. Name one path that would lock you out if missing.
  2. On a lab VM with a snapshot you can restore, enable Case 1 + Case 2, copy host keys to /persist, switch, reboot, ssh in. Confirm the host key warning does not appear.
  3. sudo touch /root/junk && sudo reboot and confirm /root/junk is gone.
  4. Add /var/lib/desk-api to directories, deploy a dummy file there, reboot, confirm it survived. Remove it from the list on a second experiment and confirm it dies.