Source Filtering

Updated

September 12, 2026

Source Filtering

src = ./. hashes the README, .git, and result. The boring default is: lib.fileset (or cleanSourceWith) so only the files the build reads are in the FOD, especially in a monorepo.

Mental model

The source tarball is an input. Extra files → extra hash → extra rebuilds → extra cache misses.

Tool When
lib.cleanSource Drop .git, result, editor junk. Fast default.
lib.cleanSourceWith Plus a custom filter
lib.fileset.toSource Union of paths; preferred on 26.05
nix-gitignore.gitignoreSource Follow .gitignore 
repo/
  apps/desk-api/   ← fileset
  docs/            ← not in src
  README.md        ← not in src

Never raw src = ./.; on a repo root for a leaf package. Include lockfiles (go.sum, Cargo.lock, package-lock.json). Exclude docs and .git.

Worked examples

Case 1: cleanSource

Save as filter.nix:

# filter.nix
{ pkgs ? import <nixpkgs> { } }:

pkgs.stdenv.mkDerivation {
  pname = "desk-src-demo";
  version = "1.0";
  src = pkgs.lib.cleanSource ./.;
  buildPhase = "echo ok > $out";
  installPhase = "true";
}
nix-build filter.nix
echo "# noise" >> README.md
nix-build filter.nix --dry-run

If README is filtered, the second command prints an empty “will be built” list. If gcc appears, README is still in src.

Case 2: fileset union

Save as fileset.nix:

# fileset.nix
{ pkgs ? import <nixpkgs> { }, lib ? pkgs.lib }:

pkgs.buildGoModule {
  pname = "desk-api";
  version = "1.0.0";
  src = lib.fileset.toSource {
    root = ./.;
    fileset = lib.fileset.unions [
      ./main.go
      ./go.mod
      ./go.sum
    ];
  };
  vendorHash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
}

A CSS change under ./web does not rebuild this derivation. Forgetting go.sum makes go complain — add it. fileset is stricter than gitignore.

lib.fileset.gitTracked ./. is “what git would ship,” then subtract:

# fileset-git.nix fragment
src = lib.fileset.toSource {
  root = ./.;
  fileset = lib.fileset.difference
    (lib.fileset.gitTracked ./.)
    (lib.fileset.unions [ ./docs ./target ./.github ]);
};

gitTracked needs a .git at eval — fine in the repo, not inside a fetchFromGitHub unpack that has no .git. For a fetched src, union the files you need; do not call gitTracked.

Case 3: Exclude by name

# exclude.nix fragment
{
  src = pkgs.lib.cleanSourceWith {
    src = ./.;
    filter = path: type:
      let base = baseNameOf path; in
      !(builtins.elem base [ "README.md" "docs" ".envrc" "result" ]);
  };
}

The filter itself must be deterministic. Do not readDir a changing cache dir. Do not use builtins.currentTime.

Case 4: gitignore

# gitignore.nix fragment
{
  src = pkgs.nix-gitignore.gitignoreSource [ ] ./.;
}

Only as good as .gitignore. If you commit data.bin that the build does not need, it still busts. Prefer fileset when you can name the files. One style per repo — not both fighting.

Case 5: Prove it with --dry-run

nix-build fileset.nix
echo x >> README.md
nix-build fileset.nix --dry-run
echo x >> main.go
nix-build fileset.nix --dry-run

README edit: empty will-build list. main.go edit: the Go derivation rebuilds. That is the filter working.

nix path-info $(nix-build fileset.nix --no-out-link --argstr ignore 0 2>/dev/null; nix-instantiate fileset.nix)

Or inspect the src path from nix show-derivation and confirm .git is absent.

drv=$(nix-instantiate fileset.nix)
nix show-derivation "$drv" | jq -r '.[].inputSrcs[]'

No .git, no target/, no node_modules. If README.md is in inputSrcs, the fileset still includes it — tighten the union.

The trap

The trap is filtering out go.sum / Cargo.lock. The FOD hash of vendors depends on the lock. Include lockfiles. Exclude docs and .git.

The other trap is filterSource that depends on builtins.currentTime or readDir of a changing cache dir. The filter itself must be deterministic.

A third: gitTracked on a fetchFromGitHub tree (no .git). A fourth: mixing gitignoreSource and fileset so nobody knows which filter won.

The boring rule

  • Never raw src = ./. on a repo root for a leaf package.
  • fileset on 26.05 when you can name the files. difference + gitTracked in a git checkout.
  • Lockfiles in; READMEs, target/, .git out.
  • --dry-run after a docs-only edit.
  • One filter style per repo (fileset or gitignore, not both fighting).

Try this

  1. Case 1 with and without cleanSource; --dry-run after a README edit.
  2. fileset of a single main.go; add a sibling file; --dry-run.
  3. Forget go.sum in the fileset; watch go complain; add it.
  4. nix show-derivation the src input; confirm .git is absent.