NixOS containers
NixOS containers
Goal: Understand isolation options on NixOS—especially declarative containers.* (nspawn-style) and light OCI awareness—and run one declarative container (or clearly document why your lab uses a pure NixOS service instead).
This chapter is not “Docker first.” Prefer a native NixOS module when it exists. Containers help for upstream images, multi-version isolation, or hostile workloads—not as a default personality. Pin remains NixOS 26.05.
Why this chapter exists
Ecosystem pressure says “everything in containers.” NixOS offers stronger default stories for many services: one store, one firewall story, one secrets story. Still, you need literacy so you can refuse containers for good reasons—or adopt them deliberately.
| Tool | Isolation style |
|---|---|
| NixOS systemd services | Same host, units, store |
containers.<name> |
systemd-nspawn guest with NixOS config |
| OCI (Podman/Docker) | Image-based; next chapter |
| VMs / k3s | Heavier; optional k3s chapter |
Theory 1 — When containers help on NixOS
| Use container | Prefer native module |
|---|---|
| Vendor only ships image | PostgreSQL, Caddy, Prometheus in nixpkgs |
| Need older/newer app than nixpkgs | Package it when feasible |
| Separate failure domain lightly | One more systemd service is enough |
| Multi-tenant sketch | Single-user lab |
| Hostile untrusted binary | Still not a full security boundary—see Theory 4 |
Decision micro-script
Is there a maintained NixOS module?
yes → use it
no → Can I package it?
yes → package + native service
no → OCI (next chapter) or nspawn NixOS guest (this chapter)
Theory 2 — containers.<name> (NixOS nspawn)
NixOS can define ephemeral/declarative containers. The guest is a nested NixOS module graph—not a Dockerfile.
# modules/services/demo-container.nix
{
containers.demo = {
autoStart = true;
privateNetwork = true;
hostAddress = "192.168.100.1";
localAddress = "192.168.100.2";
config = { config, pkgs, ... }: {
system.stateVersion = "26.05";
networking.firewall.allowedTCPPorts = [ 80 ];
services.nginx = {
enable = true;
virtualHosts."localhost" = {
root = pkgs.writeTextDir "index.html" "<h1>container</h1>";
};
};
};
};
}| Option idea | Meaning |
|---|---|
autoStart |
Start at boot |
privateNetwork |
Separate network namespace |
hostAddress / localAddress |
veth pair addressing |
bindMounts |
Host paths into guest |
config |
Nested NixOS module system |
ephemeral |
Lose rootfs state on stop (when supported) |
Exact networking options evolve—verify against 26.05 options docs if attributes differ.
Pros / cons
| Pros | Cons |
|---|---|
| Declarative guest config | Nested complexity / longer eval |
| Shares host store often | Not full VM isolation |
| Good teaching tool | Debug requires container tooling |
| Rebuild updates guest like host | Nested stateVersion still required |
sudo machinectl list
sudo nixos-container status demo 2>/dev/null || true
sudo nixos-container root-login demo 2>/dev/null || trueTheory 3 — OCI awareness (without next-chapter depth)
OCI images: layers + config + entrypoint. On NixOS you may:
- Pull images with Podman/Docker (next chapter)
- Build images from Nix (
dockerTools—packaging awareness later) - Run rootless vs rootful
An image is not a NixOS module. It is a blob with its own update story, CVE surface, and entrypoint that may ignore your host hardening.
NixOS guest (containers.*) |
OCI image |
|---|---|
| Config is Nix | Config is image + run flags |
| Updates via host rebuild | Updates via pull/tag discipline |
| Same module options as host | Ad-hoc env vars / volumes |
| Fits flake locks naturally | Needs digest pins for honesty |
Theory 4 — Security boundaries (honest)
| Boundary | Strength |
|---|---|
| Process on host | Weak isolation |
systemd hardening (ProtectSystem, …) |
Better, still host kernel |
| nspawn container | Stronger namespaces |
| OCI with user namespaces | Better if configured; easy to misconfigure |
| VM | Stronger (hardware virt) |
| Separate machine | Operational isolation |
Do not claim “container = secure” without profile hardening and a patch process for images.
Hardening checklist (guest)
- Minimal packages in guest
- Firewall on guest and publish policy on host
- No unnecessary bind mounts of
/or secret dirs - No privileged flags unless justified in writing
Theory 5 — State and containers
| Pattern | Use |
|---|---|
| Ephemeral guest | Stateless demos, rebuild-friendly |
Bind /var/lib/app from host |
Persist data; backup host path |
| Named volume (OCI) | Next chapter territory |
| Store-only content | Ideal for pure static sites |
Mounting database dirs into random containers without care is a good way to corrupt data.
Theory 6 — Relation to proxy / firewall
[client] → host:443 (caddy/nginx)
↓
192.168.100.2:80 (container)
Every publish updates docs/ports.md.
| Mistake | Fix |
|---|---|
| Open guest port on public interface | Proxy only; firewall deny direct |
Forget ports.md |
Fix immediately |
| Double NAT confusion | Draw the path once |
Theory 7 — Rebuild UX and nested eval
sudo nixos-rebuild build --flake .#lab -L| Friction vs native service | Why |
|---|---|
| Longer eval | Nested module system |
| Harder journal navigation | Machine names / nspawn units |
| Networking options maze | Worth a diagram |
Theory 8 — Feature flag for demos
{ config, lib, ... }:
let cfg = config.myLab.demoContainer; in {
options.myLab.demoContainer.enable = lib.mkEnableOption "demo nspawn container";
config = lib.mkIf cfg.enable {
containers.demo = { /* ... */ };
};
}Default off on constrained hosts.
Worked example — Prefer native, demo container optional
Path A (valid): skip long-lived containers; write a decision record “native modules suffice for the service slice.”
Path B: run the containers.demo nginx guest and curl via host networking setup you chose.
Both paths require written judgment. Path A is not a skip of the chapter—it is an evidence-backed refusal.
Exercises
Exercise 1 — Decision record
In docs/isolation.md:
# Isolation choices
Service slice:
- proxy: native module
- postgres: native module
- container: yes/no because …
Boundary strength needed:
Threat model (1 paragraph):
When we would revisit containers:Exercise 2 — Explore host tools
command -v nixos-container || true
command -v machinectl || true
systemctl list-units 'systemd-nspawn@*' 2>/dev/null | head || trueExercise 3 — One declarative container or dry config
Implement containers.demo or write the full Nix config with enable false if you truly cannot run nspawn. Prefer running when possible.
sudo nixos-rebuild switch --flake .#lab
sudo nixos-container list 2>/dev/null || sudo machinectl listExercise 4 — Reach the guest service
Curl from host to guest IP or port. Document the path in docs/isolation.md.
curl -sS http://192.168.100.2/ | headExercise 5 — Compare rebuild UX
Change guest index.html content via Nix; rebuild; note friction vs native services.nginx on host.
Exercise 6 — When you would use OCI instead
Write three bullets for the Podman chapter: cases where an upstream image beats nspawn NixOS guest.
Exercise 7 — Firewall / ports doc
Update docs/ports.md for any published path. If Path A, document that no extra ports were opened.
Exercise 8 — Nested stateVersion
Confirm guest sets system.stateVersion = "26.05" (or your real pin).
Exercise 9 — Journal path
journalctl -M demo -b --no-pager 2>/dev/null | tail || trueExercise 10 — Bind mount caution note
Write two sentences: what you would never bind-mount into a guest.
Exercise 11 — Disable cleanly
If demo only, disable container; rebuild; confirm resources freed.
Exercise 12 — Eval time measurement
time nixos-rebuild build --flake .#labNote cost of nested modules.
Exercise 13 — Diagram
ASCII diagram of client → proxy → container (or N/A for Path A).
Exercise 14 — Threat model honesty
One paragraph: nspawn is not a VM; when you would use a real VM instead.
Common gotchas
| Symptom / mistake | What to do |
|---|---|
Nested stateVersion missing |
Set in guest config |
| Networking black hole | Start simpler; read options |
| Assuming Docker API | That’s the Podman chapter |
| Container as default for Postgres | Prefer native module |
| Forgotten port documentation | Update ports.md |
| Nested eval errors | nixos-rebuild build; read trace |
| Bind-mounting secrets world-readable | Fix modes; sops on host only |
| Expecting VM-level isolation | Adjust threat model |
| Path A without write-up | Decision record required |
Checkpoint
- Isolation decision record written
- Understand nspawn-style
containers.*vs OCI - One container ran or justified skip with config draft
- Networking/ports documented
- Comparison note vs native services
- Preview of OCI use cases written
- Nested stateVersion correct if enabled
- Honest security boundary language
Journal (optional)
Write three personal gotchas before continuing.