Day 38 — TLS front door

Updated

July 30, 2026

Day 38 — TLS front door

Stage IV · ~4h
Goal: Put a TLS reverse proxy in front of a simple backend using ACME (Let’s Encrypt or lab alternative)—pick Caddy or Nginx, not both, as the house reverse-proxy pattern.

Note

Public ACME needs a public DNS name pointing at your lab. If you cannot, use internal TLS, mkcert, or Caddy/Nginx self-signed lab modes—and document the deviation. The pattern still counts.

Why this day exists

Stage IV services should not all expose their own TLS stacks. The production pattern:

Internet/LAN clients
    → :443 reverse proxy (TLS terminates)
        → 127.0.0.1:app-port (HTTP plain on localhost)

One certificate story, one cipher/policy place, one set of firewall holes (80/443).

Theory 1 — Choose one proxy

Proxy Strengths Notes
Caddy Automatic HTTPS UX, simple config Great default for labs
Nginx Ubiquitous, explicit ACME via security.acme + nginx module

Pick one for the rest of Stage IV. Write the choice in docs/proxy.md.

Theory 2 — ACME on NixOS

NixOS security.acme (used heavily with nginx) or Caddy’s built-in ACME:

Concern Practice
Email Real email for Let’s Encrypt notices
Domains You control DNS
HTTP-01 Port 80 reachable from internet
DNS-01 Provider token via sops (not plaintext)
Staging Use LE staging while debugging rate limits

Secrets

ACME account keys are managed on disk by the tooling—do not invent plaintext private keys in the flake. DNS provider tokens: sops-nix (Day 34).

Theory 3 — Caddy pattern (NixOS)

# modules/services/proxy-caddy.nix
{ pkgs, ... }:
{
  networking.firewall.allowedTCPPorts = [ 80 443 ];

  services.caddy = {
    enable = true;
    # Example virtual host — replace domain
    virtualHosts."app.example.invalid".extraConfig = ''
      encode gzip
      reverse_proxy 127.0.0.1:8080
    '';
  };
}

Caddy often obtains certs automatically when DNS is real. For invalid/lab domains, configure accordingly or use a local-only site block for HTTP lab testing.

Static site alternative:

services.caddy.virtualHosts."static.example.invalid".extraConfig = ''
  root * ${pkgs.writeTextDir "index.html" "<h1>lab</h1>"}
  file_server
'';

(Adjust to current Caddy module options on 26.05 if attribute names differ slightly—verify with option search.)

Theory 4 — Nginx + ACME pattern

# modules/services/proxy-nginx.nix
{
  networking.firewall.allowedTCPPorts = [ 80 443 ];

  security.acme = {
    acceptTerms = true;
    defaults.email = "you@example.invalid";
  };

  services.nginx = {
    enable = true;
    recommendedProxySettings = true;
    recommendedTlsSettings = true;
    virtualHosts."app.example.invalid" = {
      enableACME = true;
      forceSSL = true;
      locations."/" = {
        proxyPass = "http://127.0.0.1:8080";
        proxyWebsockets = true;
      };
    };
  };
}

Theory 5 — Backend for the day

Minimal backend options:

  1. Static files via the proxy itself
  2. python -m http.server style temporary (imperative only for proof—prefer declarative)
  3. Tiny declarative service:
systemd.services.lab-backend = {
  wantedBy = [ "multi-user.target" ];
  serviceConfig = {
    DynamicUser = true;
    ExecStart = "${pkgs.python3}/bin/python3 -m http.server 8080 --bind 127.0.0.1";
    WorkingDirectory = "${pkgs.writeTextDir "index.html" "<h1>backend</h1>"}";
  };
};

Bind 127.0.0.1 so the firewall need not open 8080.

Theory 6 — Failure modes

Symptom Checks
ACME fail DNS A/AAAA, port 80, rate limits, email/terms
502 bad gateway Backend down; wrong port; SE/socket
Cert works, app not proxyPass URL
Works on host, not remote Firewall 80/443; cloud SG 
curl -vI https://app.example.invalid
journalctl -u caddy -b --no-pager | tail -50
# or nginx / acme services

Theory 7 — Update ports.md

Port Service
80 ACME HTTP-01 / redirect
443 HTTPS proxy
8080 localhost backend only

Lab 1 — Pick proxy and document

Write docs/proxy.md with Caddy or Nginx and domain plan (public / LAN / self-signed).

Lab 2 — Localhost backend

Add declarative backend on 127.0.0.1:8080. Prove:

curl -sS http://127.0.0.1:8080/ | head

From another machine, connection should fail if not firewalled open.

Lab 3 — Enable proxy module

Import proxy module; open 80/443; rebuild.

sudo nixos-rebuild switch --flake .#lab

Lab 4 — TLS path

If public DNS: complete ACME; curl -I https://your.domain.

If not: configure self-signed or internal CA approach; curl -k for lab proof; document how production would differ.

Lab 5 — Headers and hygiene (light)

Enable recommended TLS/proxy settings (nginx flags above, or Caddy defaults). Confirm you do not open backend port publicly.

Lab 6 — Break/fix

Stop backend; observe 502; start backend; recover. Journal the unit names.

Theory 8 — Certificate storage and renewal

ACME clients store account/certs under state dirs (often under /var/lib). That state is not in the Nix store.

Event Result
System generation rollback Software/config may roll back; certs usually remain
Wipe /var Certificates gone; re-issue (rate limits!)
Domain change New cert path; update proxy vhost

Back up ACME state on anything beyond a disposable lab, or accept re-issuance cost.

Theory 9 — HTTP → HTTPS redirect

# nginx: forceSSL + enableACME typically redirects
# caddy: automatic HTTPS often redirects once certs work

Test both:

curl -I http://app.example.invalid
curl -I https://app.example.invalid

Expect redirect or secure connection—not a permanent plain HTTP production path.

Worked example — Internal TLS without public ACME

# Conceptual lab path — exact options differ by proxy:
# - generate mkcert/local CA on admin machine
# - place cert/key via sops-nix into /run/secrets
# - point proxy ssl_certificate paths at those files

Document production delta: “In prod we would use ACME DNS-01 / HTTP-01; lab uses internal CA.”

Lab 7 — ports.md + proxy.md dual-write

Ensure both docs agree on:

Item Value
Proxy engine Caddy / Nginx
Public ports 80, 443
Backend 127.0.0.1:8080
TLS mode ACME / internal / self-signed

Lab 8 — Certificate expiry awareness

echo | openssl s_client -connect 127.0.0.1:443 -servername app.example.invalid 2>/dev/null | openssl x509 -noout -dates

Even on lab self-signed, practice reading notAfter.

Common gotchas

Symptom / mistake What to do
LE rate limited Use staging; wait; fix DNS first
Both Caddy and Nginx on 443 Pick one
Backend on 0.0.0.0:8080 Bind localhost
Secrets for DNS-01 in git sops
IPv6 DNS mismatch Align AAAA or disable
Forgot port 80 for HTTP-01 Open 80 or use DNS-01

Checkpoint

  • One proxy chosen and documented
  • Backend on localhost
  • 80/443 in firewall intentionally
  • HTTPS or documented lab TLS alternative works
  • docs/ports.md updated
  • 502 break/fix once

Commit

git add .
git commit -m "day38: TLS reverse proxy front door"

Write three personal gotchas before continuing.

Tomorrow

Day 39 — Data service pattern: declarative PostgreSQL (or one DB), app role, state directories, and a backup thought exercise.