Store and daemon

Updated

July 30, 2026

Store and daemon

Goal: Explain multi-user Nix trust boundaries, GC roots, sandbox isolation on Linux, and substituter signature policy—then prove it by reading a real sandbox build log and mapping live roots on your machine.

Why this chapter exists

“It came from the cache” and “sandbox off fixed my build” are not strategies. Capstone CI, remote builders, and disaster recovery all assume you understand who can write the store, what a root pins, and what isolation actually guarantees.

Theory 1 — Single-user vs multi-user

Mode	Builder identity	Typical install	Risk profile
Single-user	Your UID owns `/nix/store`	Rare on servers; some dev installs	User compromise ≈ store compromise
Multi-user	`nix-daemon` (root) builds; clients speak to daemon	Default multi-user installer / NixOS	Trusted users can influence builds; untrusted cannot freely poison store

On NixOS you almost always run multi-user with nix-daemon.service.

systemctl status nix-daemon
ps aux | grep '[n]ix-daemon'

Clients (nix build, nixos-rebuild) send build goals to the daemon over a Unix socket (/nix/var/nix/daemon-socket/socket).

Trusted users

nix.settings.trusted-users = [ "root" "@wheel" ];
# or classic nix.conf: trusted-users = root @wheel

Trusted clients may set extra substituters, import unsigned paths (depending on config), and change certain build parameters. Do not put random CI runners in trusted-users without understanding that.

Allowed users

allowed-users gates who may talk to the daemon at all. Default is often *.

Threat sketches

Actor	Without trusted	With trusted
Unprivileged shell user	Can request builds of known paths; limited conf changes	May configure extra caches, more build power
Compromised CI job as trusted	Can push/pull trust boundaries	Treat as root-adjacent for store policy
Stolen laptop user session	Profile + user roots	Same + any trusted abilities

Theory 2 — Store paths, validity, and roots

Path anatomy (recall)

/nix/store/<hash>-<name>

The hash fingerprints the derivation inputs (input-addressed world—the default you live in). The CA derivations chapter contrasts content-addressed experimental paths.

Validity

A path is valid if the daemon registered it (built or substituted). Query:

nix path-info /nix/store/…-hello-…
nix path-info -r /nix/store/…-hello-…   # closure
nix path-info -S /nix/store/…           # size
nix path-info --json /nix/store/… | head

GC roots

Garbage collection deletes unreferenced valid paths. Roots include:

Root class	Examples
Profiles	`/nix/var/nix/profiles/system`, per-user profiles
Generations	System generation symlinks
Result symlinks	`./result`, `./result-1` from `nix build`
Runtime roots	Running processes’ mapped files (when gcroots scanning enabled)
Explicit roots	`nix-store --add-root`, GC root dirs under `/nix/var/nix/gcroots`

nix-store --gc --print-roots | head -50
# modern:
nix store gc --dry-run   # careful; prefer print-roots first
ls -la /nix/var/nix/gcroots/
ls -la /nix/var/nix/profiles/system* | head

Teaching point: deleting a git repo does not free its build products if a profile or result link still roots them.

Disk full playbook (operator)

df -h /nix
nix-store --gc --print-dead | head
# review old system generations before deleting
sudo nix-collect-garbage -d   # aggressive; know what you destroy

Prefer generation retention policy on lab hosts over panic GC mid-build.

Theory 3 — Sandbox (Linux)

Why sandbox

Builds should see:

declared inputs only
no network (except fixed-output derivations with hashed output)
clean env, controlled /dev, empty /build

So “works on my laptop” impurities become hard errors.

Mechanisms (Linux)

Rough stack (details evolve; concepts stable):

Mechanism	Role
User namespaces / uid remap	Build user isolation
Mount namespaces	Private `/`, bind store inputs
PID / IPC / UTS namespaces	Process isolation
Network namespace	No ambient net
Seccomp / syscall filters	Reduce kernel attack surface (where enabled)
`sandbox-paths`	Extra allowed paths (dangerous if wide)

Config knobs

# NixOS
nix.settings.sandbox = true;          # true | false | relaxed
nix.settings.extra-sandbox-paths = [ ];
# nix.settings.sandbox-fallback = false;

Value	Meaning
`true`	Full sandbox (preferred)
`relaxed`	Some impurities allowed (debug)
`false`	Off — only for local debugging, never as silent prod default

nix show-config | grep -i sandbox
# or
nix config show | grep -i sandbox

Fixed-output derivations (FOD)

Fetchers (fetchurl, etc.) may have network if the output hash is fixed. Wrong hash → fail. This is intentional purity for source fetching.

Nested virt / KVM note

nixosTest and some builds need kvm features. That is not the same as disabling sandbox. Keep sandbox on; provision KVM correctly on the Elitebook/lab host (see front-matter labs).

Theory 4 — Substituters and signatures

Flow

Eval produces derivation goals.
Daemon checks local store.
Queries substituters (HTTP binary caches).
Verifies signatures against trusted public keys.
Registers paths if valid.

Trust policy

nix.settings = {
  substituters = [
    "https://cache.nixos.org"
    "https://myorg.cachix.org"
  ];
  trusted-public-keys = [
    "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
    "myorg.cachix.org-1:…"
  ];
};

Misconfiguration	Outcome
Substituter without key	Paths refused (or require trust elevation)
Key without care	You accept binaries from that signer as “as good as build”
`require-sigs = false`	Dangerous; only for isolated experiments

On NixOS, nix.settings.require-sigs should stay true in normal life.

Signing your own cache (awareness)

CI often builds → signs → pushes to Cachix/Attic/Harmonia. Capstone hardening wires this into CI; this chapter only requires why the public key is in your flake/host config.

Theory 5 — Reading build logs

nix build nixpkgs#hello --rebuild   # force local rebuild if substitutes exist
nix log /nix/store/…-hello-….drv
# or after failure:
nix log .#packages.x86_64-linux.default

Sandbox-related failure fingerprints:

Log signal	Likely cause
`Network access disabled` / connection errors in non-FOD	Impure net need
`No such file or directory` for `/usr/...`	Hardcoded FHS path
Permission denied under `/nix/store`	Trying to mutate inputs
Hash mismatch	FOD impurity or wrong expected hash
`cannot build … because … is not valid`	Missing input; corrupted store

strace on daemon builds is advanced and rarely needed; prefer log literacy first.

Theory 6 — Daemon lifecycle on NixOS

systemctl cat nix-daemon.service
systemctl restart nix-daemon   # after nix.conf / nix.settings changes when needed
journalctl -u nix-daemon -e --no-pager | tail -100

When settings appear “ignored”: multi-user means daemon config, not only user ~/.config/nix/nix.conf (unless allowed). Prefer declarative nix.settings on NixOS hosts.

Worked example — Trust interview (self)

Answer in writing:

Who can talk to the daemon?
Who is trusted?
Which substituters and which keys?
Is sandbox on?
What is your oldest system generation still rooted?
What would you change before exposing a shared builder on the LAN?

This interview becomes Capstone TRUST notes.

Lab 0 — Workspace

mkdir -p ~/lab/nixos/internals/store-daemon
cd ~/lab/nixos/internals/store-daemon

Work on a machine with multi-user Nix (NixOS lab VM ideal).

Lab 1 — Map live roots

nix-store --gc --print-roots 2>/dev/null | tee roots.txt | head -80
readlink -f /run/current-system || true
ls -la /nix/var/nix/profiles/ | tee profiles.txt

In ROOTS.md, classify 10 roots into: system generation, user profile, result link, gcroots explicit, other. Explain what would free a path you care about.

Acceptance: 10 classified roots; one “how I would free X” paragraph.

Lab 2 — Daemon & trust surface

systemctl cat nix-daemon.service | tee daemon-unit.txt
nix config show | tee nix-config.txt
grep -E 'trusted|substitut|sandbox|require-sigs|allowed' nix-config.txt

Answer in TRUST.md:

Who is trusted?
Which substituters and keys?
Is sandbox on?
What would you change before exposing a shared builder?

Lab 3 — Sandbox log literacy

Force a small local build and capture the log:

nix build nixpkgs#hello --rebuild -L 2>&1 | tee hello-rebuild.log
# find drv / log:
nix log nixpkgs#hello | tee hello-nix-log.txt

If substitutes always win and --rebuild isn’t available in your version, build a tiny local derivation:

# tiny.nix
{ pkgs ? import <nixpkgs> { } }:
pkgs.runCommand "lab-tiny" { } ''
  echo hello > $out
''

nix build -f tiny.nix -L 2>&1 | tee tiny.log

Annotate 5 lines in the log (phases, sandbox mentions, store paths).

Acceptance: Annotated log snippet in notes.

Lab 4 — Optional impurity demo (VM only)

Warning

Do this only on a disposable VM. Re-enable sandbox afterward.

Note current sandbox setting.
Temporarily set sandbox = false via nix.settings or a one-off documented experiment.
Observe that impure references may start “working.”
Restore sandbox = true and rebuild the host.
Record the moral in TRUST.md.

Prefer reading docs + logs over leaving sandbox off.

Lab 5 — Signature refusal mental model

Write a short scenario (no need to actually serve malware):

Cache C serves path P signed by key K_bad. Your host only trusts K_official.

What should the daemon do? What if you are a trusted-user and force import? Document answers—tie to Capstone CI signing later.

Lab 6 — Elitebook / KVM lab acceptance

On the lab host (metal or primary VM):

# free space for store + images
df -h / /nix 2>/dev/null || df -h /
# nested virt if host is itself a VM
# egrep -i 'vmx|svm' /proc/cpuinfo | head

Acceptance criteria (lab readiness):

nix-daemon active
sandbox = true
require-sigs true (or documented exception)
≥15% free on store filesystem before heavy rebuilds
You know how to open hypervisor console if SSH dies

Common gotchas

Symptom	What to do
`cannot connect to daemon`	`nix-daemon` down; socket perms; single-user mismatch
GC deleted something “needed”	Missing root; reinstall profile / rebuild system
Build needs network mid-compile	Package not pure; fix derivation, don’t disable sandbox forever
`untrusted substituter`	Key missing or not trusted for your user
Rootful Docker “fixed” Nix	Different problem; don’t conflate
Filling disk	Roots + old generations; GC hygiene chapter
Settings ignored	Restart daemon; use declarative `nix.settings`

Checkpoint

Multi-user vs single-user explained
ROOTS.md with classified roots
TRUST.md with trusted-users, keys, sandbox
One build log annotated
Sandbox left enabled on lab host
Lab free-space / daemon acceptance ticked
Three personal gotchas

Commit

git add .
git commit -m "lab(nixos): store daemon sandbox and substituter trust"

Write three personal gotchas before continuing.

→ Nix performance — measure eval vs build, tune jobs/cores, justify settings.

Store and daemon

Why this chapter exists

Theory 1 — Single-user vs multi-user

Trusted users

Allowed users

Threat sketches

Theory 2 — Store paths, validity, and roots

Path anatomy (recall)

Validity

GC roots

Disk full playbook (operator)

Theory 3 — Sandbox (Linux)

Why sandbox

Mechanisms (Linux)

Config knobs

Fixed-output derivations (FOD)

Nested virt / KVM note

Theory 4 — Substituters and signatures

Flow

Trust policy

Signing your own cache (awareness)

Theory 5 — Reading build logs

Theory 6 — Daemon lifecycle on NixOS

Worked example — Trust interview (self)

Lab 0 — Workspace

Lab 1 — Map live roots

Lab 2 — Daemon & trust surface

Lab 3 — Sandbox log literacy

Lab 4 — Optional impurity demo (VM only)

Lab 5 — Signature refusal mental model

Lab 6 — Elitebook / KVM lab acceptance

Common gotchas

Checkpoint

Commit

Next