Day 6 — Users, Networking & System Packages

Updated

July 30, 2025

Day 6 — Users, Networking & System Packages

Day 14 — Users & access

Stage II · ~4h
Goal: Declare users and remote access properly—SSH keys as the default remote path, non-root admin via wheel, and no long-lived plaintext password habits in the flake.

Why this day exists

Most early NixOS grief is lockouts and sloppy auth: root password only, keys never deployed, secrets pasted into world-readable store paths. Fix access patterns before opening the host to a network.

Theory 1 — Declarative users

users.users.alice = {
  isNormalUser = true;
  extraGroups = [ "wheel" "networkmanager" ];
  openssh.authorizedKeys.keys = [
    "ssh-ed25519 AAAA… comment"
  ];
};
Field Role
isNormalUser Normal account defaults (home, shell group, …)
isSystemUser Service accounts
extraGroups Group membership
uid / group Pin when needed
shell e.g. pkgs.zsh
hashedPassword / initialHashedPassword Password auth
openssh.authorizedKeys.keys SSH public keys

Theory 2 — Root vs wheel admin

Pattern Notes
Root login over SSH Discourage; often disable
User in wheel + sudo Preferred admin path
Passwordless sudo Convenient lab; tighten later 
security.sudo.wheelNeedsPassword = true; # or false for lab convenience
services.openssh.settings = {
  PermitRootLogin = "no";
  PasswordAuthentication = false;
};

Exact option paths can vary slightly by release—prefer services.openssh.settings.* on modern NixOS; verify with docs/man configuration.nix / nixos-option.

Theory 3 — Passwords and the store

Approach Store risk
initialPassword = "secret" Password may land in store/world-readable eval — lab only, rotate
hashedPassword = "…" Hash still visible in store; better than plaintext but not a secret manager
SSH keys only Public keys are public; private keys never in flake
sops-nix / agenix later Stage IV

House rule now: SSH public keys in git are fine; private keys and live passwords are not.

Generate hash on the machine if you need password console login:

mkpasswd -m sha-512
# or openssl passwd -6

Theory 4 — SSH service basics

services.openssh = {
  enable = true;
  ports = [ 22 ];
  settings = {
    PasswordAuthentication = false;
    KbdInteractiveAuthentication = false;
    PermitRootLogin = "no";
  };
};

Firewall must allow the port (Day 15)—if SSH dies after firewall day, check both.

Theory 5 — Host keys vs user keys

Key type Purpose
SSH host keys Identify the machine to clients
User authorized keys Authenticate users to the machine
Your laptop private key Stays on client

Host keys live under /etc/ssh and are part of machine identity—backup/restore stories later.

Theory 6 — Groups you will actually use

Group Common reason
wheel sudo
networkmanager NM control (if NM)
docker / podman Containers later—grant carefully
video / input Desktop labs

Least privilege: do not dump every group on the admin user “just in case.”

Theory 7 — Home directories and mutability

Declarative users create homes per policy. Files inside homes are state. Rolling back system generations does not reset ~/.ssh edits. Home Manager (Stage III) later manages dotfiles; today raw user access only.

Worked examples bank

Example A — Admin user with key

users.users.alice = {
  isNormalUser = true;
  extraGroups = [ "wheel" ];
  openssh.authorizedKeys.keys = [
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExampleKeyMaterial alice@laptop"
  ];
};

Example B — Disable password SSH

services.openssh.enable = true;
services.openssh.settings.PasswordAuthentication = false;

Example C — Console password without storing plaintext

users.users.alice.hashedPassword = "$6$…"; # from mkpasswd

Lab 1 — Generate a lab keypair on the client

On your workstation (not necessarily the VM):

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_nixlab -C "nixlab"
cat ~/.ssh/id_ed25519_nixlab.pub

Copy the public key into the flake. Never commit the private key.

Lab 2 — Wire user + SSH into flake

Edit hosts/lab/configuration.nix:

  1. Admin user with wheel
  2. openssh.authorizedKeys.keys
  3. SSH enabled
  4. Prefer PasswordAuthentication = false once key login works
  5. PermitRootLogin = "no" when ready
cd ~/nixos-config
sudo nixos-rebuild switch --flake .#lab

From client:

ssh -i ~/.ssh/id_ed25519_nixlab alice@<vm-ip>

Lab 3 — Verify identity paths

On host:

whoami
groups
sudo -v
ss -ltn | grep -E ':22|:…'

On client: confirm you did not need a password for SSH (if disabled).

Lab 4 — Lockout drill (console required)

Snapshot first.

  1. Misconfigure authorized key (truncate key).
  2. Rebuild.
  3. Confirm SSH fails.
  4. Fix via console, rebuild.

Write the recovery path on your rollback card.

Lab 5 — Audit flake for secret smells

cd ~/nixos-config
rg -n "initialPassword|password\s*=" || true
rg -n "BEGIN OPENSSH PRIVATE KEY" || true

Remove plaintext habits. Document any temporary lab exceptions.

Theory 8 — mutableUsers and the declarative contract

users.mutableUsers = false; # strong: only flake defines users/passwords
# users.mutableUsers = true;  # default-ish: passwd changes may persist outside rebuild
Setting Effect
mutableUsers = true Imperative passwd / useradd can drift
mutableUsers = false Users/passwords must come from config (hashed)

For a lab that teaches flakes as SoT, consider false once SSH keys work—know that password changes then require a rebuild.

Theory 9 — Authorized keys file layout

users.users.alice.openssh.authorizedKeys.keys = [ "ssh-ed25519 AAAA… alice@laptop" ];
# or
# users.users.alice.openssh.authorizedKeys.keyFiles = [ ./keys/alice.pub ];

Prefer one key per line, full public key material, no wrapping. A truncated key fails silently as “Permission denied (publickey).”

Client config hygiene

Host nixlab
  HostName 192.0.2.10
  User alice
  IdentityFile ~/.ssh/id_ed25519_nixlab
  IdentitiesOnly yes

IdentitiesOnly yes avoids offering the wrong key and hitting MaxAuthTries.

Worked example D — System user for a service (preview)

users.users.labbot = {
  isSystemUser = true;
  group = "labbot";
  description = "Lab automation user";
};
users.groups.labbot = {};

Do not put service bots in wheel. Day 17 will run units as dedicated users.

Lab 6 — users.mutableUsers experiment (optional, snapshot)

  1. Note current mutableUsers
  2. Set explicitly in flake
  3. Rebuild
  4. Document whether passwd changes survive the next rebuild

Revert if the policy is too strict for console-only recovery yet.

Lab 7 — SSH hard fail checklist

From client after a failed login:

ssh -vvv -i ~/.ssh/id_ed25519_nixlab alice@<vm-ip> 2>&1 | tail -40

On host:

sudo journalctl -u sshd -n 50 --no-pager
ls -la /home/alice/.ssh 2>/dev/null || true
# authorized_keys may be managed under /etc/ssh/authorized_keys.d on NixOS

Write the actual authorized_keys path you observed—NixOS often does not use only ~/.ssh/authorized_keys.

Common gotchas

Symptom Theory
Key in flake but SSH password still works PasswordAuthentication still on
Permission denied (publickey) Wrong user, wrong key, typo in key line
Locked out Console + rollback
Password in git history Rotate; rewrite history if needed; prefer never
Root SSH only Add wheel user before disabling root

Checkpoint

  • Declarative non-root admin user
  • SSH key login works
  • Password SSH disabled (or explicitly justified)
  • Root SSH disabled or justified
  • No private keys in repo
  • Lockout recovery tested on console

Commit

cd ~/nixos-config
git add .
git commit -m "day14: declarative users ssh keys only"

Tomorrow

Day 15 — Networking & firewall. Interface management (networkd vs NetworkManager), addressing, and opening only what you need.

Day 15 — Networking & firewall

Stage II · ~4h
Goal: Configure lab networking deliberately—pick networkd or NetworkManager, know your address, and run a default-deny-ish firewall that still allows SSH.

Why this day exists

A flake host that “has SSH in config” but blocked by firewall, or random DHCP with no notes, is not operable. Networking is part of the system closure story.

Theory 1 — Two common network stacks on NixOS

Stack Typical use Module vibes
systemd-networkd Servers, VMs, simple static systemd.network, networking.useNetworkd
NetworkManager Laptops, Wi-Fi, desktop networking.networkmanager.enable

Do not enable both casually without understanding conflicts. For a server-like VM, networkd or simple DHCP via networking.* is enough.

Classic DHCP simplicity

networking.useDHCP = false;
networking.interfaces.ens3.useDHCP = true; # interface name from `ip a`

Or global patterns depending on release defaults—inspect generated hardware and ip a.

Theory 2 — Host identity

networking.hostName = "lab";
# networking.domain = "example.local";

Hostname affects prompts, certificates later, and mental ops. Keep flake attr and hostname documented (Day 12).

Theory 3 — Firewall model

NixOS firewall (iptables/nftables backend depending on config) is commonly:

networking.firewall = {
  enable = true;
  allowedTCPPorts = [ 22 ];
  allowedUDPPorts = [ ];
  # allowedTCPPortRanges = [ { from = 8000; to = 8010; } ];
};
Idea Practice
Default deny inbound Enable firewall
Open only needed ports List explicitly
Per-interface rules Advanced; later
Service modules open ports Some set firewall rules when enabled

Always ensure SSH port is allowed before testing remote-only access.

Theory 4 — Interface names

Predictable names (enp0s3, ens18, eth0) depend on virt hardware.

ip -br a
networkctl status   # if networkd
nmcli device        # if NM

Hard-coding the wrong interface → no network after switch. Prefer DHCP-on-all or verified names.

Theory 5 — Static addressing (optional lab)

networking.interfaces.ens3.ipv4.addresses = [{
  address = "192.168.56.10";
  prefixLength = 24;
}];
networking.defaultGateway = "192.168.56.1";
networking.nameservers = [ "1.1.1.1" "9.9.9.9" ];

Use only when you understand the hypervisor network. Wrong gateway = silence.

Theory 6 — DNS and resolution

resolvectl status  # systemd-resolved often in play
getent hosts cache.nixos.org

Broken DNS looks like “Nix is broken” during rebuilds (substituters).

Theory 7 — Firewall vs service modules

Example: enabling services.openssh does not always mean you thought about firewall—on many configs you still list port 22 (or rely on module defaults). Verify:

sudo iptables -L -n 2>/dev/null | head
# or
sudo nft list ruleset | head
ss -ltn

Day 37 hardens further; today: correctness + minimal surface.

Worked examples bank

Example A — Server-like DHCP + SSH

{
  networking.hostName = "lab";
  networking.firewall.enable = true;
  networking.firewall.allowedTCPPorts = [ 22 ];
  services.openssh.enable = true;
}

Example B — Explicit interface DHCP

{
  networking.useDHCP = false;
  networking.interfaces.ens3.useDHCP = true;
}

Example C — Temporary debug open (lab only)

# networking.firewall.allowedTCPPorts = [ 22 80 443 ];

Remove ports you do not need.

Lab 1 — Discover ground truth

On the lab host:

ip -br a
ip route
cat /etc/hostname
resolvectl status 2>/dev/null | head

Record interface name(s), IPv4, default route.

Lab 2 — Declare firewall explicitly

In flake config:

  1. networking.firewall.enable = true
  2. Allow SSH port only
  3. Rebuild (test recommended if remote)
sudo nixos-rebuild test --flake .#lab
# verify ssh still works
sudo nixos-rebuild switch --flake .#lab

Lab 3 — Port audit

ss -ltnup
# map each listener to a reason

Fill table in notes:

Port Process Declared in config? Keep?
22 sshd yes yes

Lab 4 — Controlled break (console ready)

Snapshot.

  1. Remove 22 from allowedTCPPorts while SSH is your only remote path.
  2. switch (expect lockout if no console).
  3. Fix via console; restore port.

If you are console-only already, simulate by checking nft/iptables counters and documenting the mistake.

Lab 5 — Network README snippet

Add to docs/network.md:

  • Hypervisor network mode (NAT/bridge/host-only)
  • How to find VM IP from host
  • Firewall policy summary
  • DNS notes

Theory 8 — networking.nftables awareness (26.05)

Modern NixOS often prefers nftables as the firewall backend. You rarely write raw rules by hand on day 15; you set module options and inspect:

sudo nft list ruleset | head -80
# legacy path still appears on some configs:
sudo iptables -L -n 2>/dev/null | head || true
Practice Avoid
Module options + rebuild Hand-editing live nft rules as SoT
Document backend in docs/network.md Assuming iptables forever

Theory 9 — Extra hosts and lab DNS

networking.extraHosts = ''
  127.0.0.1 lab.local
  10.0.0.5  sibling.lab
'';

Useful for multi-VM labs and reverse-proxy names later (Day 38). Keep it small; real DNS arrives with ACME.

Theory 10 — networking.firewall + rebuild safety pattern

Remote-only operators should:

sudo nixos-rebuild test --flake .#lab
# new shell: ssh still works?
sudo nixos-rebuild switch --flake .#lab

test activates without making the generation the boot default—slightly safer while iterating firewall. Console access remains the real safety net.

Worked example D — networkd file shape (server VM)

{
  networking.useNetworkd = true;
  networking.useDHCP = false;
  systemd.network.networks."10-wan" = {
    matchConfig.Name = "en*";
    networkConfig.DHCP = "yes";
  };
}

Interface match patterns reduce rename pain vs hard-coding ens3 only—verify on your virt NIC names.

Lab 6 — Firewall rule ↔︎ listener map

Build a three-column table in docs/network.md:

Listener (ss) Firewall open? Config source
:22 yes allowedTCPPorts / ssh module

Any listener without a story is either localhost-only (OK) or a future Stage IV ticket.

Lab 7 — DNS failure drill (safe)

getent hosts cache.nixos.org
# temporarily wrong nameserver in a *test* generation if you can console-recover

Document: rebuilds that need substitutes fail hard when DNS dies mid-flight—order network fixes carefully.

Common gotchas

Symptom Theory
Rebuild can’t fetch DNS/route broken mid-change
SSH timeout after firewall Port not allowed
Interface rename after hardware change Update config
Both NM and networkd fighting Pick one model
useDHCP defaults surprise Read options for 26.05

Checkpoint

  • Interface + IP documented
  • Firewall enabled with explicit SSH allow
  • Port audit table written
  • Recovery from firewall mistake understood
  • Committed network docs

Commit

cd ~/nixos-config
git add .
git commit -m "day15: networking firewall explicit ssh"

Tomorrow

Day 16 — Packages & programs. environment.systemPackages vs programs.* modules—install daily tools declaratively without random nix profile drift.

Day 16 — Packages & programs

Stage II · ~4h
Goal: Install software declaratively on the host—know when to use environment.systemPackages versus programs.* modules, and keep ad-hoc profiles from becoming the real SoT.

Why this day exists

New NixOS users nix-env / nix profile install a zoo, then wonder why rebuilds do not reproduce the machine. Declarative packages are how the host stays honest.

Theory 1 — environment.systemPackages

environment.systemPackages = with pkgs; [
  vim
  git
  curl
  htop
  jq
];
Property Meaning
On PATH system-wide For all users (typical)
Closure in system gen Rolled back with system
No service config Just binaries/libs

This is the blunt instrument—and it is correct for many CLI tools.

Theory 2 — programs.* modules

Many packages have first-class modules:

programs.git.enable = true;
programs.zsh.enable = true;
programs.tmux.enable = true;
# programs.neovim.enable = true; # if present

Modules may:

  • Install the package
  • Drop config snippets
  • Set shells in /etc/shells
  • Enable related services

Prefer programs.* when you need integration, not only a binary.

Discovery

# search options (host with nixos, or online search.nixos.org)
man configuration.nix  # huge
# or
nixos-option programs.git 2>/dev/null

Online: https://search.nixos.org/options (select 26.05 when available).

Theory 3 — Decision table

Need Prefer
Just a CLI on PATH environment.systemPackages
Shell as login shell + vendor config programs.zsh / programs.fish / programs.bash
Git system-wide with config hooks programs.git and/or HM later
Language toolchains for one project devShell (Day 8/28), not systemPackages
Desktop app packages or specialized modules
Always-on service services.* (Day 17)

Theory 4 — Closure discipline (light)

Every package on the system increases:

  • Download size
  • Eval/build time
  • Attack surface
  • Cognitive load
Smell Fix
Entire language toolchains system-wide devShell / HM
Three editors “just in case” Pick one for system
Random nix profile duplicates Remove; declare in flake

Theory 5 — Overlays and pins (awareness only)

You might see:

nixpkgs.overlays = [ … ];

or pinned packages. Stage III/V cover overlays properly. Today: stick to nixpkgs 26.05 attribute names from your lock.

Theory 6 — Unfree and allow flags

Some packages need:

nixpkgs.config.allowUnfree = true;

Or per-package predicates. Only enable when you understand licensing implications for your lab.

Theory 7 — System vs user packages

Layer Mechanism
System environment.systemPackages, programs.*
User (later) Home Manager home.packages
Project nix develop shells
Imperative nix profile — minimize

Stage III moves dotfiles and user CLIs to HM; today system layer only.

Worked examples bank

Example A — Baseline admin tools

{ pkgs, ... }: {
  environment.systemPackages = with pkgs; [
    vim
    git
    curl
    wget
    htop
    tmux
    jq
    ripgrep
    tree
  ];
}

Example B — programs integration

{
  programs.zsh.enable = true;
  users.users.alice.shell = pkgs.zsh;
}

Example C — Search package attr

nix search nixpkgs ripgrep

Lab 1 — Declare your daily CLI set

List 8–15 tools you truly want on the lab host. Add them via environment.systemPackages. Rebuild switch. Verify each:

command -v rg
command -v jq
# …

Lab 2 — Convert one package to programs.*

Pick something with a module (git, zsh, tmux, …):

  1. Remove from systemPackages if redundant
  2. Enable programs.<name>
  3. Rebuild
  4. Note extra behavior (completions, config files)

Lab 3 — Anti-profile audit

nix profile list 2>/dev/null || true
# nix-env -q 2>/dev/null || true

If tools exist only imperatively, either:

  • migrate into flake, or
  • remove them

Document the end state: flake owns system tools.

Lab 4 — Size awareness

nix path-info -Sh /run/current-system
# optional deeper:
nix path-info -rS /run/current-system | sort -n | tail

Write one sentence on the cost of your package list.

Lab 5 — Split comments in config

Organize packages with comments:

environment.systemPackages = with pkgs; [
  # editors
  vim
  # net
  curl
  # observability
  htop
];

Commit as readable policy, not a dump.

Theory 8 — environment.defaultPackages trap

NixOS may install a small default package set. Know it exists so you do not double-declare or fight removals:

# advanced / intentional only
# environment.defaultPackages = [ ];

Prefer adding what you need over emptying defaults on day one unless you understand the fallout (e.g. nano/perl expectations in scripts).

Theory 9 — nix-shell vs system packages vs flakes

Want Tool
Permanent host CLI environment.systemPackages / programs.*
Project toolchain nix develop / direnv (later)
One-off try nix shell nixpkgs#foo / nix run

Do not promote every experiment into the system closure.

Worked example D — Conditional unfree single package

{
  nixpkgs.config.allowUnfreePredicate = pkg:
    builtins.elem (pkgs.lib.getName pkg) [ "vscode" ]; # example name only
}

Narrower than global allowUnfree = true when you must pull one unfree tool onto a mostly free lab.

Lab 6 — programs.* option archaeology

# on the lab host
nixos-option programs.zsh 2>/dev/null | head -40
# or use search.nixos.org with channel 26.05

List three options under a programs.* module that are not just “install the binary.”

Lab 7 — Remove one redundant package

Find a package present both via systemPackages and a programs.* enable (or profile). Collapse to one path. Rebuild. Confirm still on PATH.

Common gotchas

Symptom Theory
Package name ≠ binary name Check pkgs attr vs meta.mainProgram
Unfree errors allowUnfree policy
Tool missing after rebuild Not in config; was profile-only
Giant system closure Too many packages / wrong layer
Shell not changing users.users.*.shell not set

Checkpoint

  • Daily tools declared in flake
  • At least one programs.* used
  • Imperative profile not required for lab tools
  • Rough system path size noted
  • Config comments organized

Commit

cd ~/nixos-config
git add .
git commit -m "day16: systemPackages and programs modules"

Tomorrow

Day 17 — systemd via Nix. One custom service + timer generated from modules—units as code.