Building a Network Forensics Lab Environment

Author

K19G

Updated

October 31, 2026

Building a Network Forensics Lab Environment

Conducting rigorous research, validation, and training in behavior-based network forensics requires an isolated, reproducible testbed. A production-grade forensics lab must fulfill three demanding criteria:

Absolute Safety and Containment: Toxic payloads, live adversary emulation, and malware C2 traffic must never escape the sandbox or contaminate production infrastructure or the public Internet.
High-Fidelity Realism: Malicious behavior stands out only when immersed in realistic background enterprise traffic (DNS queries, TLS handshakes, user web browsing, periodic backups, active directory authentication). Emulating an attack in a sterile, silent network yields artificial, brittle detection models that fail in production.
Forensic Reproducibility: Every test run must be reproducible from code, generating byte-exact evidence trails with synchronized timestamps and defensible chain of custody.

┌────────────────────────────────────────────────────────────────────────┐
│              High-Fidelity Forensics Lab Architecture                  │
└────────────────────────────────────────────────────────────────────────┘
                    ┌──────────────────────────────┐
                    │ Adversary Emulation Engine   │
                    │ (Caldera / Atomic Red Team)  │
                    └──────────────┬───────────────┘
                                   │ (Attacker Actions)
                                   ▼
┌──────────────────────┐    ┌──────────────┐    ┌────────────────────────┐
│ Benign Background    ├───>│ Isolated Lab ├───>│ Victim / Target Hosts  │
│ Traffic Generator    │    │ L2/L3 Fabric │    │ (Linux, Windows Server)│
│ (Containerlab/T-Rex) │    └──────┬───────┘    └────────────────────────┘
└──────────────────────┘           │
                                   │ Optical / Virtual TAP
                                   ▼
                    ┌──────────────────────────────┐
                    │ Out-of-Band Forensic Sensor  │
                    │ (Zeek / Arkime / Suricata)   │
                    └──────────────┬───────────────┘
                                   │ Immutable Telemetry (SHA-256)
                                   ▼
                    ┌──────────────────────────────┐
                    │ Evidence Vault / SIEM WORM   │
                    └──────────────────────────────┘

Lab Infrastructure as Code (Containerlab Topology)

Using Containerlab, the forensic lab is codified in a declarative YAML topology. The architecture segregates management access, victim enclaves, adversary attack positions, and out-of-band forensic monitoring:

name: forensics-lab

topology:
  nodes:
    # --- Virtual TAP & Forensic Capture Sensor ---
    sensor:
      kind: linux
      image: ghcr.io/arkime/arkime:latest
      network-mode: none
      binds:
        - ./data/forensics:/data/pcap:rw

    # --- Router / Core Gateway ---
    core-gw:
      kind: linux
      image: frrouting/frr:v8.5.2
      binds:
        - ./config/daemons:/etc/frr/daemons
        - ./config/frr.conf:/etc/frr/frr.conf

    # --- Victim Enclave ---
    victim-srv:
      kind: linux
      image: debian:bookworm-slim
      exec:
        - ip route add default via 10.10.10.1

    victim-client:
      kind: linux
      image: alpine:latest
      exec:
        - ip route add default via 10.10.10.1

    # --- Adversary Emulation Node ---
    adversary:
      kind: linux
      image: mitre/caldera:latest
      exec:
        - ip route add default via 172.16.100.1

  links:
    - endpoints: ["victim-srv:eth1", "core-gw:eth1"]
    - endpoints: ["victim-client:eth1", "core-gw:eth2"]
    - endpoints: ["adversary:eth1", "core-gw:eth3"]
    # Mirrored monitor link connected directly to sensor
    - endpoints: ["core-gw:eth4", "sensor:eth1"]

Port Mirroring and Virtual TAP Configuration

In Linux-based software switches (ovs-vsctl or Linux bridge tc), network traffic between adversary and victim segments must be mirrored to the out-of-band forensic sensor without introducing packet drops or modifying packet headers.

Using Linux Traffic Control (tc) with the mirred action:

#!/usr/bin/env bash
# Mirror all ingress and egress packets on eth1 (Victim Server) to eth4 (Sensor)

# Add ingress qdisc
tc qdisc add dev eth1 ingress

# Mirror all incoming packets on eth1 to eth4
tc filter add dev eth1 ingress protocol all u32 match u32 0 0 \
  action mirred egress mirror dev eth4

# Add egress qdisc
tc qdisc add dev eth1 root handle 1: prio

# Mirror all outgoing packets on eth1 to eth4
tc filter add dev eth1 parent 1: protocol all u32 match u32 0 0 \
  action mirred egress mirror dev eth4

This configuration ensures the forensic sensor receives byte-exact copies of all traffic, including malformed frames, fragmented packets, and truncated TCP handshakes.

Benign Background Traffic Generation

The biggest failure of intrusion detection research is the “sterile lab fallacy”—testing malware in an empty network where any packet is anomalous. Realistic forensics requires a dense background of legitimate enterprise traffic:

1. Web and API Noise (HTTP/TLS)

We script curl worker bots inside Docker containers simulating human browsing patterns with randomized Poisson-distributed arrival intervals:

#!/usr/bin/env python3
import time, random, requests

SITES = [
    "https://api.github.com/zen",
    "https://en.wikipedia.org/wiki/Main_Page",
    "http://10.10.10.10/intranet/index.html",
    "http://10.10.10.10/api/v1/metrics"
]

def generate_benign_traffic():
    session = requests.Session()
    session.headers.update({"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36"})
    while True:
        target = random.choice(SITES)
        try:
            session.get(target, timeout=3)
        except Exception:
            pass
        # Exponential inter-arrival distribution simulating human reading pauses
        sleep_time = random.expovariate(1.0 / 4.5)  # Mean 4.5 seconds
        time.sleep(sleep_time)

if __name__ == "__main__":
    generate_benign_traffic()

2. DNS and NTP Telemetry

Legitimate network segments emit regular DNS lookups (reverse PTR, internal A/AAAA queries) and NTP time synchronization bursts. Running local BIND9 and Chrony instances inside the lab populates DNS caches and generates standard protocol background telemetry.

Adversary Emulation Frameworks

Rather than running uncontrolled, live worm payloads that risk escaping, behavioral network forensics relies on adversary emulation frameworks that execute deterministic, parameterized TTPs mapped to the MITRE ATT&CK framework:

Framework	Core Capability	Behavioral Forensics Utility
MITRE Caldera	Automated adversary emulation with autonomous agents and planners	Simulates multi-stage APT campaigns; measures telemetry drift across lateral movements
Atomic Red Team	Library of simple, actionable, single-technique test scripts	Tests atomic behavioral detectors (e.g., T1046 Network Service Discovery, T1071 C2 Protocols)
Scapy Custom Scripts	Python packet synthesis and raw socket manipulation	Synthesizes custom protocol anomalies, covert timing channels, and DGA beacons

Emulating a Beaconing C2 Channel with Jitter

To test behavioral anomaly detection against command-and-control beacons, execute a parameterized Scapy probe that mimics Cobalt Strike HTTP beaconing with \(20\%\) jitter:

#!/usr/bin/env python3
import time, random, requests

C2_URL = "http://172.16.100.50/submit.php"
BASE_INTERVAL = 30.0  # seconds
JITTER = 0.20         # 20% jitter

while True:
    jitter_delta = BASE_INTERVAL * JITTER * (random.uniform(-1.0, 1.0))
    sleep_duration = BASE_INTERVAL + jitter_delta
    time.sleep(sleep_duration)
    
    try:
        # Simulate small check-in heartbeat payload
        requests.post(C2_URL, data={"id": "session-44a", "status": "idle"}, timeout=2)
    except Exception:
        pass

Defensible Evidence Handling in the Lab

To support forensic reproducibility and peer review:

Automated Run Manifests: Every lab execution automatically logs git commit hashes, kernel versions, Containerlab topology definitions, and environment variables into a signed manifest.json.
Synchronized Clocks: All container hosts and sensor nodes synchronize to a dedicated local master clock using PTP (ptp4l) or Chrony, achieving sub-millisecond timestamp alignment across PCAPs and syslogs.
Automated Evidence Sealing: At test completion, a post-execution script terminates traffic, calculates SHA-256 hashes across all captured PCAPs, locks file permissions to read-only (0440), and creates a detached GPG signature.

Chapter Summary

A network forensics lab requires strict network isolation, realistic background noise generation, and deterministic adversary emulation.
Containerlab combined with Linux tc mirred port mirroring delivers a high-speed, reproducible testing fabric as code.
Background traffic generation prevents the “sterile lab fallacy,” forcing behavioral algorithms to separate low-signal adversary TTPs from high-volume benign noise.
Automated manifests, timestamp synchronization, and cryptographic evidence sealing guarantee scientific and judicial reproducibility.