Day 40 — nixos-containers / OCI

Updated

July 30, 2026

Day 40 — nixos-containers / OCI

Stage IV · ~4h
Goal: Understand isolation options on NixOS—especially declarative containers.* (nspawn-style) and light OCI awareness—and run one declarative container (or clearly document why your lab uses a pure NixOS service instead).

Note

This day is not “Docker first.” Prefer a native NixOS module when it exists (Days 38–39). Containers help for upstream images, multi-version isolation, or hostile workloads—not as a default personality. Pin remains NixOS 26.05.

Why this day exists

Ecosystem pressure says “everything in containers.” NixOS offers stronger default stories for many services: one store, one firewall story, one secrets story. Still, you need literacy so you can refuse containers for good reasons—or adopt them deliberately.

Tool Isolation style
NixOS systemd services Same host, units, store
containers.<name> systemd-nspawn guest with NixOS config
OCI (Podman/Docker) Image-based; Day 41 modules
VMs / k3s Heavier; Day 43 optional

Skipping this day leaves a blind spot: every vendor README assumes Docker, and you will paste it without knowing what you sacrificed.

Theory 1 — When containers help on NixOS

Use container Prefer native module
Vendor only ships image PostgreSQL, Caddy, Prometheus in nixpkgs
Need older/newer app than nixpkgs Simple Go binary you package in Stage V
Separate failure domain lightly One more systemd service is enough
Multi-tenant sketch Single-user lab
Hostile untrusted binary Still not a full security boundary—see Theory 4

Decision micro-script

Is there a maintained NixOS module?
  yes → use it (Days 38–39)
  no  → Can I package it (Stage V)?
          yes → package + native service
          no  → OCI (Day 41) or nspawn NixOS guest (today)

Theory 2 — containers.<name> (NixOS nspawn)

NixOS can define ephemeral/declarative containers. The guest is a nested NixOS module graph—not a Dockerfile.

# modules/services/demo-container.nix
{
  containers.demo = {
    autoStart = true;
    privateNetwork = true;
    hostAddress = "192.168.100.1";
    localAddress = "192.168.100.2";
    config = { config, pkgs, ... }: {
      system.stateVersion = "26.05";
      networking.firewall.allowedTCPPorts = [ 80 ];
      services.nginx = {
        enable = true;
        virtualHosts."localhost" = {
          root = pkgs.writeTextDir "index.html" "<h1>container</h1>";
        };
      };
      # minimal users/sshd usually not needed for service-only guests
    };
  };
}
Option idea Meaning
autoStart Start at boot
privateNetwork Separate network namespace
hostAddress / localAddress veth pair addressing
bindMounts Host paths into guest
config Nested NixOS module system
ephemeral Lose rootfs state on stop (when supported)

Exact networking options evolve—verify against your 26.05 options docs (man configuration.nix / search.nixos.org) if attributes differ.

Pros / cons

Pros Cons
Declarative guest config Nested complexity / longer eval
Shares host store often Not full VM isolation
Good teaching tool Debug requires container tooling
Rebuild updates guest like host Nested stateVersion still required 
sudo machinectl list
sudo nixos-container status demo 2>/dev/null || true
sudo nixos-container root-login demo 2>/dev/null || true

Theory 3 — OCI awareness (without Day 41 depth)

OCI images: layers + config + entrypoint. On NixOS you may:

  • Pull images with Podman/Docker (Day 41)
  • Build images from Nix (dockerTools, nixery, etc.—later packaging awareness)
  • Run rootless vs rootful

Today: know that an image is not a NixOS module. It is a blob with its own update story, CVE surface, and entrypoint that may ignore your host hardening.

NixOS guest (containers.*) OCI image
Config is Nix Config is image + run flags
Updates via host rebuild Updates via pull/tag discipline
Same module options as host Ad-hoc env vars / volumes
Fits flake locks naturally Needs digest pins for honesty

Theory 4 — Security boundaries (honest)

Boundary Strength
Process on host Weak isolation
systemd hardening (ProtectSystem, …) Better, still host kernel
nspawn container Stronger namespaces
OCI with user namespaces Better if configured; easy to misconfigure
VM Stronger (hardware virt)
Separate machine Operational isolation

Do not claim “container = secure” without profile hardening and a patch process for images. Kernel exploits ignore your Docker Compose feelings.

Hardening checklist (guest)

  • Minimal packages in guest
  • Firewall on guest and publish policy on host
  • No unnecessary bind mounts of / or secret dirs
  • No privileged flags unless justified in writing

Theory 5 — State and containers

Guest state can live in:

  • Container root filesystem areas
  • Bind mounts from host paths

Bind mounts must respect secrets and backup stories. Mounting /var/lib/postgresql into a random container without care is a good way to corrupt data.

Pattern Use
Ephemeral guest Stateless demos, rebuild-friendly
Bind /var/lib/app from host Persist data; backup host path
Named volume (OCI) Day 41 territory
Store-only content Ideal for pure static sites

Theory 6 — Relation to proxy / firewall

If a container listens only on a private network, the host proxy may reverse-proxy into it—or you publish a port on the host. Every publish updates docs/ports.md.

[client] → host:443 (caddy/nginx)
              ↓
         192.168.100.2:80 (container)
Mistake Fix
Open guest port on public interface Proxy only; firewall deny direct
Forget ports.md Stage IV gate will bite
Double NAT confusion Draw the path once

Theory 7 — Rebuild UX and nested eval

Changing guest config requires host evaluation of nested modules. Failures look like host rebuild failures with deeper traces.

# build only (no switch) to catch nested errors early
sudo nixos-rebuild build --flake .#lab -L
Friction vs native service Why
Longer eval Nested module system
Harder journal navigation Machine names / nspawn units
Networking options maze Worth a diagram
Worth it when? Real isolation need, not fashion

Worked example — Prefer native, demo container optional

Path A (valid for gate): skip long-lived containers; write a decision record “native modules suffice for Stage IV slice.”

Path B: run the containers.demo nginx guest and curl via host networking setup you chose.

Both paths require written judgment. Path A is not a skip of the day—it is an evidence-backed refusal.

Lab 1 — Decision record

In docs/isolation.md:

# Isolation choices

Stage IV slice services:
- proxy: native module
- postgres: native module
- container: yes/no because …

Boundary strength needed:
Threat model (1 paragraph):
When we would revisit containers:

Lab 2 — Explore host tools

command -v nixos-container || true
command -v machinectl || true
systemctl list-units 'systemd-nspawn@*' 2>/dev/null | head || true
man systemd-nspawn 2>/dev/null | head || true

Record which tools exist on your 26.05 lab host.

Lab 3 — One declarative container or dry config

Implement containers.demo or write the full Nix config in-repo under modules/services/demo-container.nix with enable false and a comment—only if you truly cannot run nspawn in your environment (e.g. restricted VM). Prefer actually running it when possible.

sudo nixos-rebuild switch --flake .#lab
sudo nixos-container list 2>/dev/null || sudo machinectl list

Lab 4 — Reach the guest service

Depending on networking mode, curl from host to guest IP or port. Document the path in docs/isolation.md.

# example privateNetwork style
curl -sS http://192.168.100.2/ | tee ~/lab/90daysofx/02-nixos/day40/curl.txt

Lab 5 — Compare rebuild UX

Change guest index.html content via Nix; rebuild; observe whether container picks up changes cleanly. Note friction vs native services.nginx on host (time to change, journal path, failure modes).

Lab 6 — When you would use OCI instead

Write three bullets for Day 41: cases where an upstream image beats nspawn NixOS guest.

Lab 7 — Firewall / ports doc

Update docs/ports.md (or create it) for any published path. If Path A (no container), document that no extra ports were opened.

Common gotchas

Symptom / mistake What to do
Nested stateVersion missing Set in guest config ("26.05")
Networking black hole Start with simpler network mode; read options
Assuming Docker API That’s Day 41
Container as default for Postgres Prefer native module
Forgotten port documentation Update ports.md
Nested eval errors nixos-rebuild build; read trace from deepest file
Bind-mounting secrets world-readable Fix modes; consider sops on host only
Expecting VM-level isolation Adjust threat model; use real VM if needed

Checkpoint

  • Isolation decision record written
  • Understand nspawn-style containers.* vs OCI
  • One container ran or justified skip with config draft
  • Networking/ports documented
  • Comparison note vs native services
  • Preview of Day 41 use cases

Commit

git add .
git commit -m "day40: nixos-containers isolation pattern"

Write three personal gotchas before continuing.

Tomorrow

Day 41 — Podman/Docker modules: OCI on NixOS with module discipline; rootless notes; compare to Day 40.