Testing Strategies for Nix Configurations

Testing Strategies for Nix Configurations

nixos-rebuild switch on prod is not a test. The boring default is: nix flake check on every PR, pkgs.testers.runNixOSTest for services that listen, and a VM that is not the laptop.

Mental model

Level	Command	What it proves
Eval	nix eval, module types	The expression is a value
Build	nix build .#desk-api	The derivation realises
NixOS VM	runNixOSTest	A unit actually listens
Full gate	nix flake check	Every checks.* attr

A module that is mkIf false evals forever. A VM test that curls the port catches the empty wantedBy.

VM tests need kvm (and usually nixos-test) on the builder. Darwin laptops offload them to a Linux remote builder. checkPhase inside a derivation cannot hit the network — mock or vendor fixtures.

Keep PR tests under a few minutes. Nightly runs the farm.

Worked examples

Case 1: Cheap flake checks

Save as flake.nix:

# flake.nix
{
  description = "Desk checks";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";

  outputs = { self, nixpkgs }:
    let
      system = "x86_64-linux";
      pkgs = nixpkgs.legacyPackages.${system};
    in
    {
      packages.${system}.desk-api = pkgs.hello;

      checks.${system}.fmt = pkgs.runCommand "desk-fmt" { } ''
        echo ok > $out
      '';

      checks.${system}.api = self.packages.${system}.desk-api;
    };
}

nix flake check -L

-L prints logs. Failures must be visible without SSH to the runner. Replace hello with the real desk-api package.

Case 2: NixOS VM test

Save as test-desk.nix:

# test-desk.nix
{ pkgs ? import <nixpkgs> { } }:

pkgs.testers.runNixOSTest {
  name = "desk-nginx";
  nodes.server = {
    services.nginx.enable = true;
    networking.firewall.allowedTCPPorts = [ 80 ];
    system.stateVersion = "26.05";
  };
  nodes.client = {
    system.stateVersion = "26.05";
  };
  testScript = ''
    start_all()
    server.wait_for_unit("nginx.service")
    client.succeed("curl -f http://server/")
  '';
}

nix-build test-desk.nix

Needs KVM. Wire the same attr into checks.${system}.nginx so nix flake check runs it in CI.

A failing assertion:

error: client failed: curl -f http://server/

Non-zero exit. That is the gate.

Case 3: CI job

Save as .github/workflows/check.yml (shape):

# .github/workflows/check.yml
name: check
on: [pull_request]
jobs:
  flake:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: nix flake check -L

The runner must have Nix and, for Case 2, KVM (self-hosted Linux builder with supportedFeatures = [ "kvm" "nixos-test" ]). GitHub-hosted ubuntu without nested virt skips the VM check or offloads it.

Case 4: Hermetic checkPhase

# desk-api.nix fragment
{
  doCheck = true;
  checkPhase = ''
    # no curl to api.desk.internal
    go test ./...
  '';
}

If the test needs a fixture, vendor it in src. A check that needs prod credentials is not a check — it is a smoke probe you run from ops-01 after deploy.

Case 5: What not to run on every PR

A 20-node mesh test on every docs typo is how people skip CI. Split:

When	What
Every PR	fmt, .#desk-api, one small runNixOSTest
Nightly	linkFarm of images, multi-node mesh
After Colmena	live curl from ops-01

The trap

The trap is eval-only CI. nix flake check that only type-checks modules never sees wantedBy = [ ]. Pair eval with at least one VM test per listening service, or you will discover the empty unit on prod.

The boring rule

• flake check -L on every PR. VM tests when a unit listens.
• KVM on the Linux builder. 26.05 runNixOSTest.
• No network in checkPhase.
• Small PR tests; big nightly.
• A test that needs prod credentials is a probe, not a check.

Try this

1. nix flake check -L on the desk flake.
2. Run Case 2 (needs KVM). Confirm curl succeeds.
3. Add client.succeed("false") and confirm non-zero exit.
4. Add a checks attr that is just self.packages.${system}.desk-api so a broken Go build fails the PR without a VM.