Multi-Cloud Deployment Patterns

Multi-Cloud Deployment Patterns

Two clouds is two IAM models, two images, two outages. The boring default is: one cloud until you have a written reason; if you must span, one flake with two nixosConfigurations and two tofu roots — not a unified abstraction layer.

Mental model

NixOS closures are cloud-agnostic. Images and metadata are not. An AWS AMI is not a GCP image. nixos-generators emits per-format artifacts from the same modules.

modules/desk-api.nix
    ├── nixosConfigurations.desk-aws   → amazon image
    └── nixosConfigurations.desk-gcp   → gce image
tofu/aws/   state in s3://desk-tf-state/aws
tofu/gcp/   state in gcs://desk-tf-state/gcp

Do not share state files across clouds. Do not for_each {aws, gcp} in one root. A lock in the wrong cloud is an outage in the right one.

WireGuard between clouds is a mesh, not a reason to invent a “cloud-agnostic” Nix wrapper.

Worked examples

Case 1: Shared module, two hosts

Save as flake.nix:

# flake.nix
{
  description = "Desk two-cloud (only if you must)";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";

  outputs = { self, nixpkgs }:
    let
      mk = name: extra: nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          ./modules/common.nix
          extra
          { networking.hostName = name; }
        ];
      };
    in
    {
      nixosConfigurations = {
        desk-aws = mk "desk-aws" ./hosts/aws.nix;
        desk-gcp = mk "desk-gcp" ./hosts/gcp.nix;
      };
    };
}

Save as modules/common.nix:

# modules/common.nix
{
  system.stateVersion = "26.05";
  services.openssh.enable = true;
  services.openssh.settings.PasswordAuthentication = false;
  networking.firewall.enable = true;
  # desk-api service lives here
}

aws.nix / gcp.nix own disks, metadata, and the provider’s idea of a NIC. common.nix owns users, sshd, desk-api.

Case 2: Same birth, two clouds

stateVersion is birth, not channel. If both hosts are new on 26.05:

# already in common.nix
{ system.stateVersion = "26.05"; }

If desk-aws was born on 24.11, leave that host at "24.11" and only set "26.05" on the new GCP box. Copy-pasting stateVersion across clouds is how you skip a migration.

Case 3: Images from the generator, not a conversion script

nix build .#amazon
nix build .#gce

Typical artifacts (names follow your generator pin):

result/nixos-amazon-image-26.05.vhd
result/nixos-image-26.05-x86_64-linux.raw.tar.gz

Do not convert an AMI to GCE with a random script if the generator already has gce / googleComputeImage. Two formats, one module set.

Case 4: Separate tofu directories

tofu/aws/main.tf
tofu/aws/backend.tf    # bucket desk-tf-state, key desk/aws
tofu/gcp/main.tf
tofu/gcp/backend.tf    # bucket desk-tf-state, key desk/gcp

Save as tofu/aws/backend.tf:

# tofu/aws/backend.tf
terraform {
  backend "s3" {
    bucket = "desk-tf-state"
    key    = "desk/aws/terraform.tfstate"
    region = "eu-central-1"
  }
}

Different backends. Different credentials. Apply from tofu/aws never loads GCP state.

Case 5: WireGuard between clouds, not a service mesh

If the only reason for two clouds is “HA,” a WireGuard mesh (10.100.0.0/24, /32 peers) plus Colmena is enough. desk-aws is 10.100.0.11, desk-gcp is 10.100.0.12. UDP 51820 on each public NIC. Postgres still only on wg0.

Multi-cloud Kubernetes is not a first project. It is a second control plane, a second IAM, and a second way to lose DNS.

ls tofu/aws tofu/gcp
test ! -f tofu/terraform.tfstate && echo 'no shared state at tofu/ (good)'

If a single terraform.tfstate sits at tofu/, you already merged the roots. Split them before the next apply.

The trap

The trap is a “cloud-agnostic” Nix wrapper that hides AMI vs GCE differences until tofu apply. Keep the differences in aws.nix / gcp.nix where you can read them. The other trap is one terraform state that for_eachs both clouds.

The boring rule

• One cloud first. Write the sentence that justifies the second.
• Same NixOS modules; different image formats and tofu roots.
• stateVersion per host birth — not “whatever the other cloud has.”
• No shared terraform state.
• Mesh if you need routing; not a service mesh across clouds on day one.

Try this

1. List what actually differs between two clouds for the desk (image, IP, disk, IAM).
2. Draw two tofu dirs on paper. Do not merge them.
3. Build .#amazon and .#gce from one module set.
4. Write the one-sentence reason you need the second cloud — or delete the work.