Terraform boundary

Updated

July 30, 2026

Terraform boundary

Goal: Draw a clean boundary—Terraform (or OpenTofu) provisions cloud resources; NixOS owns system configuration—and produce a sketch you could implement without turning TF into a second module system. Images from the generators chapter + deploy from deploy-rs/Colmena sit on the Nix side; pin OS stories to 26.05.

Note

Full multi-cloud automation is out of scope. A sharp sketch with one example resource graph is the bar. Implement only if you already have cloud credentials and want the stretch.

Why it matters

Nix people overreach into provisioning; TF people template snowflake user-data forever. The 2026 happy path for many teams:

Terraform/OpenTofu: VPC, VM, disks, DNS, IAM, object storage
        │
        │  outputs: IP, IDs, SSH target
        ▼
Nix flake: Disko/image/deploy-rs/Colmena configure the OS

Without a boundary doc, every new hire reinvents the split—and usually badly.

Theory 1 — Responsibility split

Concern Owner Why
Cloud account / IAM TF Lifecycle & policy engines
VM size, subnet, SG TF Provider API native
Disk attach / snapshots API TF Cloud object model
Packages, services, users Nix Pure rebuild story
SSH hardening Nix Modules
App releases Nix / CI Closures
DNS record for service Often TF Or Nix + provider with care
Golden AMI/GCE image build Nix + CI Images chapter
Instance launch from image TF Provider resource

Smell: 500-line user_data shell scripts installing docker by curl.

Smell: Nix trying to create AWS VPCs with ad-hoc runCommand and keys in the store.

Theory 2 — Handoff patterns

Pattern A — Image + launch

CI: nix build image → upload AMI/GCE image
TF: instance from image ID
Nix: optional second-phase deploy-rs for fast iteration

Pattern B — Generic distro image + first deploy

TF: stock NixOS or minimal image + SSH key
deploy-rs/Colmena: converge to desired closure

Pattern C — NixOS on metal (no TF)

Disko + install + deploy-rs
TF only for adjacent cloud DNS/objects if any
Pattern Best when
A Immutable cattle, slow-changing base
B Fast module iteration
C Homelab / colo

Theory 3 — What to put in TF state vs flake lock

Artifact System of record
VPC ID, instance ID TF state (remote backend)
nixpkgs rev, module graph flake.lock + git
Deploy proof file Nix modules
Object storage bucket for backups TF create; Nix restic config consumes name
DNS API token Secret manager; neither plaintext git

Never put TF state in the Nix store. Never put flake lock “truth” only in a random S3 object without git.

Theory 4 — OpenTofu vs Terraform

For this book, treat OpenTofu and Terraform as interchangeable at the boundary level. Choose based on org license/policy. Examples may say TF generically.

# versions.tf — illustrative
terraform {
  required_version = ">= 1.6.0"
  required_providers {
    # provider pins…
  }
}

Theory 5 — Minimal resource graph (sketch)

[network]
  vpc
  subnet
  security_group (22 from admin CIDR; 80/443 public if web)
[compute]
  instance (ami/image from Nix pipeline or stock)
  eip / dns A record
[data]
  data_volume (optional)
  backup_bucket
# pseudo-HCL — not a full provider config
resource "aws_instance" "edge" {
  ami           = var.nixos_ami # produced by image pipeline
  instance_type = "t3.small"
  subnet_id     = aws_subnet.main.id
  vpc_security_group_ids = [aws_security_group.edge.id]
  key_name      = var.ssh_key_name # or cloud-init only

  tags = {
    Role = "edge"
    Nix  = "flake-managed"
  }
}

output "edge_ip" {
  value = aws_instance.edge.public_ip
}
# after apply
deploy .#edge   # hostname from TF output

Theory 6 — Secrets across the boundary

Secret Where
Cloud provider credentials TF env / CI OIDC—not flake
SSH deploy key Agent / CI secret—not world-readable
sops age keys Host persist / secure bootstrap
DB passwords sops-nix on Nix side after instance exists

TF sensitive outputs still land in state—protect state backends.

Worked example — BOUNDARY.md template

# BOUNDARY.md
## Owners
- Cloud resources: Terraform/OpenTofu in repo `infra/`
- OS/app: Nix flake in repo `nix/`

## Handoff
1. TF apply → outputs IP, instance id, bucket name
2. Update flake host inventory / sops if needed
3. deploy-rs/Colmena apply
4. Health checks

## Image pipeline
- Built by: CI job …
- Consumed by: TF var nixos_ami

## Non-goals
- TF will not install packages
- Nix will not create VPCs

## Secrets
- …

Worked example — inventory bridge

# hosts/edge/deployment-ip.nix — generated or hand-updated from TF output
{
  # Prefer automation later; honesty first
  # deployment target:
  # networking… 
}
tofu output -raw edge_ip > /tmp/edge_ip
# feed deploy hostname

Automating TF output → flake is optional stretch; document the manual bridge first.

Lab — multi-step

Suggested notes: ~/lab/nixos/ops/terraform

Step 1 — Write BOUNDARY.md

Fill the template even with zero cloud resources.

Step 2 — Draw resource graph

ASCII or mermaid for one environment (lab).

Step 3 — Choose pattern A/B/C

Record why for your context.

Step 4 — Optional implement

If credentials exist: one VM + SG + output IP; deploy Nix once.

Step 5 — User-data audit

If any user-data remains, cap it at <20 lines or delete.

Step 6 — State backend note

Where would state live? Who accesses it?

Step 7 — Failure story

“TF created VM, deploy failed”—recovery steps.

Step 8 — Stretch destroy

tofu destroy only for disposable lab; never prod without ceremony.

Exercises

  1. Assign 15 concerns to TF vs Nix vs Shared in a table.
  2. Rewrite a fictional 80-line user-data into image+modules bullets.
  3. Design IAM least privilege for CI that builds AMIs.
  4. Explain why flake lock is not TF state (one paragraph).
  5. Multi-env: how do you stop lab AMI IDs leaking into prod TF?

Common gotchas

Symptom / mistake What to do
Drift: hand-edited security group TF is source; import or recreate
Deploy to wrong IP Outputs + inventory discipline
Secrets in TF console output logs sensitive + log hygiene
NixOS firewall vs cloud SG confusion Both; document layers
AMI stale Image pipeline cadence
Two teams, no BOUNDARY.md Write it
remote-exec growth Delete; bake image

Checkpoint

  • BOUNDARY.md written
  • Resource graph sketched
  • Pattern A/B/C chosen
  • Handoff steps listed
  • Optional real TF apply or explicit skip
  • User-data policy stated
  • State backend thought through

Commit

git add .
git commit -m "ops: terraform/nix boundary sketch"

Write three personal gotchas before nixosTest.