Day 63 — Terraform boundary

Updated

July 30, 2026

Day 63 — Terraform boundary

Stage VI · ~4h (design + light lab)
Goal: Draw a clean boundary—Terraform (or OpenTofu) provisions cloud resources; NixOS owns system configuration—and produce a sketch you could implement without turning TF into a second module system. Images from Day 62 + deploy from Days 59–60 sit on the Nix side; pin OS stories to 26.05.

Note

Full multi-cloud automation is out of scope. A sharp sketch with one example resource graph is the bar. Implement only if you already have cloud credentials and want the stretch.

Why this day exists

Nix people overreach into provisioning; TF people template snowflake user-data forever. The 2026 happy path for many teams:

Terraform/OpenTofu: VPC, VM, disks, DNS, IAM, object storage
        │
        │  outputs: IP, IDs, SSH target
        ▼
Nix flake: Disko/image/deploy-rs/Colmena configure the OS

Without a boundary doc, every new hire reinvents the split—and usually badly.

Theory 1 — Responsibility split

Concern	Owner	Why
Cloud account / IAM	TF	Lifecycle & policy engines
VM size, subnet, SG	TF	Provider API native
Disk attach / snapshots API	TF	Cloud object model
Packages, services, users	Nix	Pure rebuild story
SSH hardening	Nix	Day 36
App releases	Nix / CI	Closures
DNS record for service	Often TF	Or Nix + provider with care
Golden AMI/GCE image build	Nix + CI	Day 62
Instance launch from image	TF	Provider resource

Smell: 500-line user_data shell scripts installing docker by curl.

Smell: Nix trying to create AWS VPCs with ad-hoc runCommand and keys in the store.

Theory 2 — Handoff patterns

A — Image + TF instance

CI builds image (Day 62) → upload AMI/GCE image
TF launches VM from that image
Optional: deploy-rs for day-2 changes

B — Generic distro image + first boot Nix

TF launches stock Ubuntu/NixOS cloud image
cloud-init runs nixos-rebuild from flake or nixos-anywhere / similar
Day-2: Colmena

C — nixos-anywhere / terraform modules community

Community modules exist that glue TF + NixOS installs. Treat as optional accelerators; still keep the boundary mental model.

Pattern	Best when
A	You control golden images
B	Fast bring-up; tolerate first-boot complexity
C	You accept community module churn

Theory 3 — Data that must cross the boundary

# terraform outputs
output "edge_ip" {
  value = aws_instance.edge.private_ip
}

output "edge_id" {
  value = aws_instance.edge.id
}

# Not required to auto-import—often human or generated inventory:
# hosts/edge/deployment.hostname = "10.0.1.20";

Approach	Pros	Cons
Manual inventory update	Simple	Drift
TF writes `inventory.json`	Automatable	Pipe discipline
cloud-init pulls flake ref	Flexible	Needs network/auth
SSM/agent	Ops heavy	Not Nix-native

Pick one handoff and write it in BOUNDARY.md. Two competing handoffs become three truths.

Theory 4 — Secrets across the boundary

Secret	Where
Cloud provider keys	TF env / CI OIDC; never Nix store
SSH deploy key	Agent / CI; public key in Nix
sops age keys	Host persist; not in TF state if avoidable
TF state	Encrypted backend; restricted IAM
Bootstrap passwords	Avoid; prefer keys

TF state can contain sensitive outputs—treat state as secret-bearing. Never commit terraform.tfstate.

Theory 5 — OpenTofu note

OpenTofu is a common FOSS Terraform fork in 2026 fleets. Examples below use generic HCL; prefer whatever your org standardizes. Concepts transfer 1:1 for this day’s boundary. Commands: tofu vs terraform—document which you used.

Theory 6 — What not to put in user_data

Temptation	Prefer
`apt install nginx`	Nix module
Curl-pipe installers	Package or image bake
Writing private keys into cloud-init	sops after boot / metadata carefully
Encoding entire NixOS config as bash	Flake deploy

Short cloud-init that only installs Nix or points nixos-anywhere is a gray zone—keep it tiny and versioned if you must.

Worked example — sketch only (AWS-shaped)

# infra/edge.tf (sketch)
resource "aws_instance" "edge" {
  ami           = var.nixos_ami # produced by image pipeline
  instance_type = "t3.small"
  subnet_id     = aws_subnet.private.id
  vpc_security_group_ids = [aws_security_group.edge.id]
  key_name      = aws_key_pair.lab.key_name

  tags = {
    Role         = "edge"
    ManagedBy    = "terraform"
    ConfiguredBy = "nix"
    NixOS        = "26.05"
  }
}

resource "aws_security_group" "edge" {
  name   = "edge-lab"
  vpc_id = aws_vpc.lab.id
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = [var.admin_cidr]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# BOUNDARY.md
## Provision (TF)
- VPC, subnet, SG, instance, EIP optional
- Outputs: private_ip, instance_id

## Configure (Nix)
- roles.web, profiles.hardened
- deploy with colmena --on edge once IP known

## Forbidden
- apt install in user_data
- embedding age private keys in TF
- nix store cloud credentials

Lab — multi-step

Suggested workspace: ~/lab/90daysofx/02-nixos/day63

Step 1 — Draw the diagram

ASCII or markdown: resources vs modules vs deploy.

Step 2 — Write `BOUNDARY.md`

Include forbidden list and secret placement. Explicit 26.05 pin policy for OS images.

Step 3 — Inventory handoff design

Specify file format, e.g.:

{
  "edge": { "host": "10.0.1.20", "tags": ["web", "edge"] }
}

Who writes it? Who reads it (Colmena targetHost / deploy-rs hostname)?

Step 4 — Optional implement

If you have a cloud account: one VM with TF/OpenTofu + one Colmena/deploy-rs deploy. If not, stop at sketch—gate accepts sketch.

Step 5 — Failure modes table

Failure	Owner	Detection
Wrong AMI	TF/CI image	Boot fails
Wrong SG	TF	SSH timeout
Bad Nix module	Nix	activate fails
State unlock	TF	team process
Drifted inventory IP	Handoff	deploy to wrong host

Step 6 — Compare pure NixOS on metal

When TF is unnecessary (homelab bare metal), document “TF optional.” Boundary still useful for future cloud.

Step 7 — Stretch: `terraform output -json` → jq → host file

Script a generator; do not commit secrets. Dry-run only is fine.

Step 8 — Teach-back

Explain the split in 90 seconds without slides. If you mention user_data bash stacks positively, rewrite BOUNDARY.md.

Step 9 — Lifecycle matrix

Fill once for your sketch host:

Lifecycle event	TF	Nix	CI
Create VM	yes	no	optional image build first
Resize disk	yes	maybe growpart in OS	—
Change SSH policy	no	yes	—
Rotate deploy key (public)	maybe	yes	yes
Destroy VM	yes	secrets offline first	—

Step 10 — Homelab exception card

If Capstone is bare metal only, still keep BOUNDARY.md with:

## Current: no cloud
- Provision: manual / PXE / ISO (Day 62 ISO optional)
- Configure: flake + deploy-rs/Colmena
- When cloud arrives: reuse this boundary; do not invent user_data

Worked anti-patterns (recognize and refuse)

Anti-pattern A: TF file provisioners copying configs into /etc
  → day-2 drift; use Nix modules

Anti-pattern B: Nix runCommand calling aws cli with keys in env during build
  → impure builds; credentials in logs; use TF/OIDC outside the store

Anti-pattern C: Two DNS writers (Route53 TF + Nix DNS module) without owner
  → pick one writer per zone/record class

Common gotchas

Symptom / mistake	What to do
user_data megascript	Delete; bake image or nixos-anywhere
Nix store has cloud keys	Move to env/OIDC
Two sources of hostname truth	Pick TF or Nix DNS story
Applying TF as “config management”	Resist; day-2 is Nix
State file in git	Use remote backend
Sketch so vague nobody could implement	Add one concrete resource graph

Checkpoint

Boundary diagram written
BOUNDARY.md with forbidden list
Handoff inventory designed
Optional real VM or explicit sketch-only completion
Secrets placement clear

Commit

git add .
git commit -m "day63: terraform/nix boundary sketch"

Write three personal gotchas before continuing.

Tomorrow

Day 64 — nixosTest: VM tests for your modules.

Day 63 — Terraform boundary

Why this day exists

Theory 1 — Responsibility split

Theory 2 — Handoff patterns

A — Image + TF instance

B — Generic distro image + first boot Nix

C — nixos-anywhere / terraform modules community

Theory 3 — Data that must cross the boundary

Theory 4 — Secrets across the boundary

Theory 5 — OpenTofu note

Theory 6 — What not to put in user_data

Worked example — sketch only (AWS-shaped)

Lab — multi-step

Step 1 — Draw the diagram

Step 2 — Write BOUNDARY.md

Step 3 — Inventory handoff design

Step 4 — Optional implement

Step 5 — Failure modes table

Step 6 — Compare pure NixOS on metal

Step 7 — Stretch: terraform output -json → jq → host file

Step 8 — Teach-back

Step 9 — Lifecycle matrix

Step 10 — Homelab exception card

Worked anti-patterns (recognize and refuse)

Common gotchas

Checkpoint

Commit

Tomorrow

Step 2 — Write `BOUNDARY.md`

Step 7 — Stretch: `terraform output -json` → jq → host file