GC and store hygiene

Updated

July 30, 2026

GC and store hygiene

Goal: Measure store size, explain GC roots, set a retention policy for generations and CI, and run optimise/GC safely without deleting live systems.

Why this day exists

Deploy + CI + packaging fill disks silently. Out-of-disk mid-activate is a self-inflicted outage. Hygiene is an operational skill equal to deploy tooling—Capstone hosts die from full /nix as often as from bad modules.

Theory 1 — What GC deletes

Garbage collection deletes store paths that are not reachable from any GC root.

Root examples Keeps alive
/run/current-system Running NixOS closure
/nix/var/nix/profiles/* System generations, user profiles
~/.local/state/nix/profiles Modern user profiles
/nix/var/nix/gcroots/* Explicit pins
Result symlinks in cwd ./result from nix build
Indirect roots / registries Various tooling pins 
nix-store --gc --print-roots | head -n 50
nix-collect-garbage --dry-run
nix store gc --dry-run  # modern CLI shape; flags evolve—check --help

Mental model

GC root → profile/generation/result link → closure of store paths
Anything not reachable is eligible for deletion

Deleting a project directory does not free the store if roots still point at builds.

Theory 2 — Generations vs store GC

# List system generations
sudo nix-env -p /nix/var/nix/profiles/system --list-generations
# or inspect profile links
ls -l /nix/var/nix/profiles/system*

# Delete old generations then GC
sudo nix-collect-garbage --delete-older-than 14d
# aggressive: remove all old gens of profiles
# sudo nix-collect-garbage -d
Command Effect
nix-collect-garbage GC unreachable paths
--delete-older-than Drop old gens then GC
-d Aggressive old gen removal
nix store gc Modern CLI GC
nix store optimise Hardlink identical files
Important

Deleting generations removes rollback options. Keep enough to survive a bad deploy week—especially before Capstone freeze and DR drills.

User vs system profiles

Home-Manager and user nix profile installs have their own generation chains. System GC policy does not automatically equal user hygiene.

nix profile list 2>/dev/null || true
ls ~/.local/state/nix/profiles 2>/dev/null || true

Theory 3 — Policy template

Host class Generation retention Notes
Laptop 7–14 days Disk pressure
Prod VM 14–30 days + keep last N Rollback window
CI runner Daily GC; few profiles Ephemeral preferred
Builder Weekly GC; keep hot cache set Balance disk vs rebuild
Capstone lab ≥14 days through the rollback and tests chapter Rollback drills need gens 
# NixOS automatic GC example
{ ... }:
{
  nix.gc = {
    automatic = true;
    dates = "weekly";
    options = "--delete-older-than 14d";
  };
  nix.optimise.automatic = true;
  # proactive free-space GC thresholds (bytes)
  nix.settings.min-free = 10 * 1024 * 1024 * 1024; # 10 GiB
  nix.settings.max-free = 50 * 1024 * 1024 * 1024;
}

min-free triggers proactive GC when disk is low—excellent on small roots and cloud boxes.

Theory 4 — Finding what holds space

df -h /nix
nix path-info -Sh /run/current-system
nix path-info -rS /run/current-system | sort -k2 -n | tail -n 20

# Why is path alive?
nix-store --query --roots /nix/store/…-something
# Old result links (common disk vampires)
find "$HOME" -maxdepth 3 -type l \( -name result -o -name 'result-*' \) 2>/dev/null
# Largest store paths (approx; tools vary)
nix path-info -rS /run/current-system | sort -k2 -h | tail

CI-specific roots

Runners accumulate result links and old profiles if jobs are not ephemeral. Prefer clean workspaces per job; document GC for self-hosted runners in docs/CI.md.

Theory 6 — Incident patterns

Symptom First checks
No space left on device during build df -h /nix /; roots; old gens; result links
Activate failed mid-switch Free space; incomplete profile; repair with boot generation
GC “deleted my package” Missing root; rebuild; avoid bare -d in prod
Rollback missing Generations deleted by policy
Disk full only on CI Self-hosted runner hygiene

Never run destructive GC on prod without:

  1. Confirming current-system root
  2. Confirming retention window
  3. Preferring --delete-older-than over blind -d

Worked example — hygiene report script

#!/usr/bin/env bash
# scripts/store-report.sh
set -euo pipefail
echo "== disk =="
df -h /nix 2>/dev/null || df -h /
echo "== current system closure =="
nix path-info -Sh /run/current-system 2>/dev/null || true
echo "== gc roots (sample) =="
nix-store --gc --print-roots 2>/dev/null | head -n 40 || true
echo "== result symlinks =="
find "$HOME" -maxdepth 2 -type l -name 'result*' 2>/dev/null || true
echo "== system generations =="
sudo nix-env -p /nix/var/nix/profiles/system --list-generations 2>/dev/null | tail -n 15 || true
chmod +x scripts/store-report.sh
./scripts/store-report.sh | tee ~/lab/90daysofx/02-nixos/day67/before.txt

Lab — multi-step

Suggested notes: ~/lab/90daysofx/02-nixos/day67

Step 1 — Baseline metrics

Record df -h /nix (or /) and current-system size via nix path-info -Sh.

Step 2 — Map roots

Print roots; identify unexpected result links and old profiles. Screenshot or tee a sample.

Step 3 — Safe cleanup

Remove stale ./result from lab dirs; GC without aggressive -d first:

nix store gc
df -h /nix

Step 4 — Generation policy

List generations; delete only those outside your written policy window.

sudo nix-collect-garbage --delete-older-than 14d

Step 5 — Enable automatic GC on lab host

Commit nix.gc (+ optional min-free / optimise) settings; deploy or switch.

Step 6 — optimise

Run optimise; record space reclaimed if measurable.

df -h /nix | tee ~/lab/90daysofx/02-nixos/day67/before-optimise.txt
nix store optimise
df -h /nix | tee ~/lab/90daysofx/02-nixos/day67/after-optimise.txt

Step 7 — CI runner note

Add GC step for self-hosted runners or rely on ephemeral runners; document in docs/CI.md.

Step 8 — Policy doc

Write docs/STORE-HYGIENE.md (or section in RUNBOOK):

Host Retention Auto GC Notes
lab 14d weekly
laptop 7d weekly

Step 9 — Rollback still possible?

Confirm at least one previous generation remains on lab after cleanup. If not, you over-pruned—adjust policy.

Step 10 — Profile audit (user + HM)

# User profiles / modern nix profile
nix profile list 2>/dev/null | tee ~/lab/90daysofx/02-nixos/day67/user-profile.txt || true
# Home-Manager generations if present
ls -la /nix/var/nix/profiles/per-user/"$USER"/ 2>/dev/null || true
ls -la ~/.local/state/home-manager 2>/dev/null || true

Write one sentence: how user-profile retention differs from system nix.gc on this host.

Step 11 — Dry-run before aggressive delete

# Prefer dry-run / list before destroy
nix-collect-garbage --delete-older-than 30d --dry-run 2>&1 \
  | tee ~/lab/90daysofx/02-nixos/day67/gc-dry-run.txt || true

If dry-run is unavailable on your Nix version, list generations explicitly and delete only named old gens.

Worked example — min-free incident story

Disk at 98% on small cloud root:
  1) df -h /nix
  2) remove stale result links in ~ and CI workspaces
  3) nix-collect-garbage --delete-older-than 7d   # laptop policy, not prod
  4) enable nix.settings.min-free so next time GC starts earlier
  5) consider separate /nix partition next reprovision (Disko)

Record whether you hit this during Stage V/VI. Capstone hosts on 20 GiB disks fail predictably without policy.

Common gotchas

Symptom / mistake What to do
GC “does nothing” Roots still hold paths
Deleted all rollbacks Stop bare -d in prod without policy
/nix on small partition Separate partition; min-free
optimise confuses backups Hardlinks; tool configuration
Multi-user permissions GC as root for system profiles
HM gens forgotten User profile hygiene
Full disk mid-CI Ephemeral runners; job GC; slimmer checks

Checkpoint

  • Measured store before/after
  • Explained roots with real examples from your machine
  • Retention policy written and applied on lab
  • optimise run once
  • Auto GC or min-free configured
  • At least one rollback generation still present
  • User/HM profile note written
  • docs/STORE-HYGIENE.md (or RUNBOOK section) committed
  • Three personal gotchas

Commit

git add .
git commit -m "day67: GC policy and store hygiene"

Write three personal gotchas before continuing.