k3s (optional awareness)

Updated

July 30, 2026

k3s (optional awareness)

Goal: Either stand up a single-node k3s for literacy or skip cleanly with a written decision—Kubernetes is not required for this part’s service slice.

Important

If you do not need k8s for work or curiosity right now, skip. A forced k3s install can destabilize a small lab (CPU, disk, conceptual overload).

Why this chapter exists

Some learners finish a proxy + Postgres slice and ask: “Should everything move to Kubernetes?” On NixOS the honest answer is usually no—not for a one-node lab with Caddy + Postgres.

This chapter exists to:

Prevent FOMO-driven architecture
Give a safe optional on-ramp if you truly need k8s APIs
Practice skipping as a professional skill

Theory 1 — Decision rubric

Choose skip if…	Choose k3s lite if…
Service slice already works natively	You must learn kubectl for a job soon
Host is small on RAM/disk	You have ≥8–16 GB RAM lab resources
You have not finished secrets/proxy/DB solidly	Those foundations are already solid
Curiosity only, no time	You will actually deploy one manifest

Default recommendation: skip, write the decision, move on.

Theory 2 — What k3s is

k3s is a lightweight Kubernetes distribution. On NixOS:

# modules/services/k3s.nix — only if you opt in
{
  services.k3s = {
    enable = true;
    role = "server";
    # extraFlags = [ "--write-kubeconfig-mode=0644" ]; # understand security implications
  };
}

Options and defaults change—read current module docs for 26.05 before enabling.

Implications:

Area	Effect
Ports	API server, kubelet, etc.—firewall carefully
CPU/RAM	Non-trivial idle cost
Storage	Container images + etcd/sqlite data
Mental model	YAML/Helm vs NixOS modules

Theory 3 — NixOS vs k8s configuration

Concern	NixOS native	k3s
System packages	flake modules	images inside cluster
Host firewall	`networking.firewall`	+ k8s Services/NodePorts
Secrets	sops-nix on host	also k8s Secrets (different!)
Rollback	generations	cluster state is separate

Running k3s does not replace host flake discipline. You now have two control planes: NixOS + cluster.

Theory 4 — Single-node honesty

Single-node k3s is fine for learning APIs. It is not:

Multi-AZ HA
A reason to abandon PostgreSQL module for “cloud native” on one VPS without backups
Automatically more secure

Theory 5 — Minimal literacy tasks (if enabled)

kubectl get nodes
Deploy a tiny Deployment + Service
Expose via host proxy or port-forward—not random NodePort to the world
Delete the workload
Document how to disable services.k3s and reclaim resources

Theory 6 — Skip path is first-class

Skipping is documented output:

# docs/k3s-decision.md
Decision: SKIP
Date: …
Reasons:
- Service slice does not require k8s
- Host resources: …
- Will revisit if: job requirement / multi-service mesh need

That file passes the chapter.

Theory 7 — When NixOS modules beat k8s on a single node

Workload	Prefer
One reverse proxy + one DB	NixOS modules
Many identical microservices with k8s API dependency	k3s/k8s
GPU training cluster	Often specialized schedulers
Homelab “because CNCF”	Usually modules + compose/podman

NixOS is already a declarative control plane for one machine. k3s adds a second control plane—justify it.

Theory 8 — Resource floor honesty

Single-node k3s wants real RAM/CPU. On a 2 GB VM:

k3s + postgres + caddy + prometheus = thrash

Skip is the professional choice when resources are tight.

Theory 9 — Feature flag module

# modules/services/k3s.nix
{ config, lib, ... }:
let cfg = config.myLab.k3s; in {
  options.myLab.k3s.enable = lib.mkEnableOption "optional single-node k3s";
  config = lib.mkIf cfg.enable {
    services.k3s = {
      enable = true;
      role = "server";
      # extraFlags = [ "--disable traefik" ]; # if you keep host proxy as edge
    };
    # networking.firewall.allowedTCPPorts = [ 6443 ]; # admin API — lock down
  };
}

Default enable = false on the lab host.

Worked example — Disable / enable toggles

Prefer a role flag or simply do not import k3s module. Avoid half-enabled clusters.

Worked example — Tiny workload (if enabled)

# manifests/hello.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
spec:
  replicas: 1
  selector:
    matchLabels: { app: hello }
  template:
    metadata:
      labels: { app: hello }
    spec:
      containers:
        - name: hello
          image: docker.io/library/nginx:alpine
          ports:
            - containerPort: 80

kubectl apply -f manifests/hello.yaml
kubectl get pods
kubectl port-forward deploy/hello 8088:80
curl -sS http://127.0.0.1:8088/ | head

Exercises

Exercise S — Skip path (most readers)

Write docs/k3s-decision.md with SKIP + reasons
Skim what k3s would open (ports, disk) from docs—no install
List two future triggers to revisit
Commit and proceed

Time box: ~30 minutes. Do not guilt-install.

Exercise S2 — Skip artifact quality bar

Your skip note must include:

Decision (skip)
Resource or complexity reason
What you use instead (NixOS modules / podman)
Revisit criteria (“if job requires CKA lab…”)

A one-word “skipped” is not enough.

Exercise K1 — Opt-in: enable k3s

Only if rubric says go:

sudo nixos-rebuild switch --flake .#lab
sudo systemctl status k3s
sudo k3s kubectl get nodes

Kubeconfig location per module—follow module notes. Restrict access modes.

Exercise K2 — Tiny workload

Apply the hello Deployment; port-forward; curl; delete or keep intentionally.

Exercise K3 — Tear-down plan

Document:

kubectl delete -f manifests/hello.yaml
# disable services.k3s; rebuild; confirm resources freed

Update firewall if you opened API ports. Prefer API on localhost / VPN only.

Exercise K4 — Compare control planes

Write a short note: what still belongs in the NixOS flake vs what you put in the cluster.

Exercise K5 — API surface inventory (if enabled)

sudo k3s kubectl get nodes
sudo k3s kubectl get pods -A | head
ss -tlnp | grep -E '6443|10250' || true

Add every k3s port to docs/ports.md with “cluster-internal / admin-only” labels.

Exercise 6 — Resource estimate

Write RAM/disk numbers before and (if enabled) after k3s.

Exercise 7 — Secrets boundary note

One paragraph: host sops vs k8s Secrets—both needed if cluster is on.

Exercise 8 — Proxy interaction

If Traefik conflicts with Caddy/Nginx, disable one; document edge story.

Exercise 9 — Decision commit

git add docs/k3s-decision.md
git commit -m "docs: k3s optional decision recorded"

Exercise 10 — FOMO check

Write two sentences: what pressure almost made you install k3s, and why you did or did not.

Exercise 11 — Revisit calendar

If skip: pick a future review date or trigger (job change, multi-node need).

Exercise 12 — ports.md consistency

Whether skip or enable, ports.md should not claim k3s ports that do not exist.

Exercise 13 — Disable half-install

If you started and regret: disable module, rebuild, free disk images.

Exercise 14 — Part retrospective

List the service slice components you trust without k8s: proxy, DB, secrets, firewall, observability.

Common gotchas

Symptom / mistake	What to do
Guilt install before readiness	Use skip path
Opened k8s API to world	Close; bind/firewall
Disk full of images	Prune; disable k3s
Secrets only in k8s, host ignored	Host sops still matters
Two proxies fighting	One ingress story
Undocumented cluster	Decision doc mandatory either way
Thrashing small VM	Skip; free resources
“Everything in cluster”	Push back; host still owns kernel/disks

Part close-out

You should now have a service and security spine:

Capability	Chapter
Store ≠ vault	Secrets problem
Encrypt-in-git secrets	sops-nix (+ agenix compared)
Reduced attack surface	Hardening
Deny-by-default network	Firewall + ports.md
TLS edge	Reverse proxy
Durable data	PostgreSQL pattern
Isolation literacy	nspawn + OCI
Inspectability	Observability
k8s FOMO managed	Optional k3s decision

Later parts (packaging, ops/fleet, capstone) assume this discipline. If secrets still live as Nix strings or 5432 is world-open, fix that before scaling out.

k3s (optional awareness)

k3s (optional awareness)

Why this chapter exists

Theory 1 — Decision rubric

Theory 2 — What k3s is

Theory 3 — NixOS vs k8s configuration

Theory 4 — Single-node honesty

Theory 5 — Minimal literacy tasks (if enabled)

Theory 6 — Skip path is first-class

Theory 7 — When NixOS modules beat k8s on a single node

Theory 8 — Resource floor honesty

Theory 9 — Feature flag module

Worked example — Disable / enable toggles

Worked example — Tiny workload (if enabled)

Exercises

Exercise S — Skip path (most readers)

Exercise S2 — Skip artifact quality bar

Exercise K1 — Opt-in: enable k3s

Exercise K2 — Tiny workload

Exercise K3 — Tear-down plan

Exercise K4 — Compare control planes

Exercise K5 — API surface inventory (if enabled)

Exercise 6 — Resource estimate

Exercise 7 — Secrets boundary note

Exercise 8 — Proxy interaction

Exercise 9 — Decision commit

Exercise 10 — FOMO check

Exercise 11 — Revisit calendar

Exercise 12 — ports.md consistency

Exercise 13 — Disable half-install

Exercise 14 — Part retrospective

Common gotchas

Checkpoint

If skipped

If enabled

Journal (optional)

Part close-out