Capstone: optional multi-node bar

Updated

July 30, 2026

Capstone: optional multi-node bar

Goal: Feature freeze the Capstone A MVP—no new services or toys—only bugfixes, doc fixes, and checklist honesty—so disaster recovery tests a stable, tagged artifact. Optionally add a second node only if Capstone A is already green and hardware allows.

Important

After the freeze tag, the only legitimate code changes before wipe are P0 blockers that make rebuild or health checks impossible. Everything else goes in docs/POST-FREEZE.md.

Why this chapter exists

Wipe/rebuild against a moving target is chaos. Freeze creates a known baseline: annotated git tag, green CI, known gaps explicitly waived with risk. Without freeze, disaster-recovery drills measure luck, not process.

Experts freeze early relative to demos. Amateurs keep “just one more module” until the disk wipe and then invent history.

Theory 1 — What “freeze” means

Allowed Forbidden
Bugfixes for P0/P1 with issue id New apps / node roles / hostnames
Doc corrections that match code “Quick” plugin or package installs
Test stability fixes (flakes, waits) Refactors without bug id
Checklist updates (honest ticks) Starting Capstone B
Dependency bumps only if build broken Casual nix flake update
Typo-level RUNBOOK fixes New sops keys without re-documenting DR

Freeze is a social contract with yourself

If Capstone is a solo lab, freeze still matters: during DR you will be tired and tempted to “fix while rebuilding.” The tag is the line between recovering a known system and redesigning under fire.

Tag shape

git tag -a capstone-a-freeze -m "Capstone A feature freeze"
git push origin capstone-a-freeze  # if remote exists
git rev-parse capstone-a-freeze

Record tag SHA in docs/CAPSTONE-A.md.

Theory 2 — Waiver register

Every unticked Capstone A row at freeze needs one of:

Disposition Meaning
Done Proof command recorded
Waived Explicit risk + why DR still valid
Blocker Must fix before freeze tag 
# docs/POST-FREEZE.md / freeze waivers
| Item | Disposition | Risk | Owner |
|------|-------------|------|-------|
| CI cache push | Waived | Slower rebuilds | you |

Theory 3 — Optional multi-node bar

Capstone A is single-node sufficient. Multi-node is optional prestige:

Optional Description
Second guest deploy-a or similar with shared modules
Client/server test nixosTest multi-node
Distinct roles edge vs data

Elitebook 16 GB guidance

Config Advice
Host + 1 guest 4–6 GB Capstone A default
Host + 2 guests Only if you drop desktop/k3s toys
Nested heavy tests Prefer serial; lower max-jobs

Do not add a second node if it risks freeze honesty or disk pressure before DR.

Shared modules pattern

modules/common.nix
modules/roles/edge.nix
modules/roles/data.nix
hosts/lab/…
hosts/deploy-a/…   # optional

Avoid copy-paste hosts that drift.

Theory 4 — Freeze gate commands

nix flake metadata
nix build .#nixosConfigurations.lab.config.system.build.toplevel -L
nix flake check -L   # or specific checks
# deploy once more if remote path is Capstone-critical
systemctl --failed   # on lab

All green → tag. Any red → fix or waive with eyes open.

Concrete freeze checklist

  • CAPSTONE-A honest ticks
  • Waivers written
  • CI green on freeze commit
  • RUNBOOK matches freeze tag
  • Offline keys verified present
  • VM snapshot pre-dr taken
  • git tag -a capstone-a-freeze
  • POST-FREEZE.md for deferred ideas

Lab 0 — Workspace

mkdir -p ~/lab/nixos/capstone/freeze
cd /path/to/capstone-flake

Lab 1 — Honesty pass on CAPSTONE-A

For each box: done with proof, waived, or blocker. No “partial” without a sentence.

Lab 2 — Green suite at freeze

nix build .#nixosConfigurations.lab.config.system.build.toplevel -L \
  2>&1 | tee ~/lab/nixos/capstone/freeze/toplevel.log
nix flake check -L 2>&1 | tee ~/lab/nixos/capstone/freeze/check.log

Lab 3 — Tag

git status
git tag -a capstone-a-freeze -m "Capstone A feature freeze $(date -I)"
git show capstone-a-freeze --stat | tee ~/lab/nixos/capstone/freeze/tag.show

Lab 4 — Optional second node (only if A green)

  1. Snapshot first.
  2. Add host with shared modules.
  3. Deploy or rebuild.
  4. Document network and secrets recipients.
  5. If it destabilizes freeze, revert and leave as POST-FREEZE.

Lab 5 — Snapshot for DR

# virsh snapshot-create-as lab pre-dr --disk-only --atomic
# or hypervisor UI

Confirm you can revert without using DR path (sanity).

Lab 6 — Freeze announcement (to yourself)

Write 10 lines in NOTES: what is in, what is out, tag SHA, next step = DR wipe.

Common gotchas

Mistake Fix
Tagging red CI Fix or waive first
Freeze then “one more feature” POST-FREEZE.md
Multi-node thrash on 16 GB Defer
No snapshot before DR Snapshot now
Lost tag SHA Record in CAPSTONE-A

Checkpoint

  • Honest CAPSTONE-A + waivers
  • Green build/check logs
  • capstone-a-freeze annotated tag
  • pre-dr snapshot
  • Optional multi-node either done or deferred
  • Three personal gotchas

Commit

git add .
git commit -m "capstone: feature freeze readiness"
# tag after this commit if clean

Write three personal gotchas before continuing.

Next

→ Disaster recovery — wipe and rebuild from flake + keys.