Site Tunnels

Updated

September 4, 2026

Site Tunnels

Site-to-site tunnels connect islands of private addressing across a routed underlay (lab “Internet” or WAN). This chapter implements GRE and policy-based / route-based IPsec-style patterns with Linux in Containerlab, optionally fronted by FRR for routing (mid-2026 free tools).

Learning goals

By the end of this chapter you can:

Build a GRE tunnel and route site prefixes across it
Explain route-based vs policy-based VPN approaches
Run a simple WireGuard or LibreSwan/strongSwan-style lab path (tool availability permitting)
Fail underlay links and predict overlay behavior
Clamp MTU and verify with large pings and tcpdump

When site tunnels are the right tool

Use	Not the first tool for
Two branches over untrusted WAN	DC fabric east-west (use fabric overlay)
Quick lab multi-site	Massive multi-tenant scale (EVPN fabrics)
Crypto requirement	Cleartext GRE alone on hostile nets

Topology — two sites + “Internet”

h1--siteA--\                /--siteB--h2
            +---- inet ----+

name: site-tun

topology:
  nodes:
    sitea:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils tcpdump iptables wireguard-tools || true
    siteb:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils tcpdump iptables wireguard-tools || true
    inet:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils tcpdump
        - sysctl -w net.ipv4.ip_forward=1
    h1:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils
    h2:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils

  links:
    - endpoints: ["h1:eth1", "sitea:eth1"]
    - endpoints: ["h2:eth1", "siteb:eth1"]
    - endpoints: ["sitea:eth2", "inet:eth1"]
    - endpoints: ["siteb:eth2", "inet:eth2"]

Addressing plan

Object	Address
Site A LAN	192.168.10.0/24 (sitea .1, h1 .10)
Site B LAN	192.168.20.0/24 (siteb .1, h2 .10)
sitea WAN	198.51.100.1/30
inet eth1	198.51.100.2/30
siteb WAN	203.0.113.1/30
inet eth2	203.0.113.2/30
GRE outer	WAN IPs
GRE inner	10.255.255.0/30 (.1 a, .2 b)

# site A
docker exec clab-site-tun-sitea sh -c '
  sysctl -w net.ipv4.ip_forward=1
  ip addr add 192.168.10.1/24 dev eth1; ip link set eth1 up
  ip addr add 198.51.100.1/30 dev eth2; ip link set eth2 up
  ip route add 203.0.113.0/30 via 198.51.100.2
'
docker exec clab-site-tun-h1 sh -c '
  ip addr add 192.168.10.10/24 dev eth1; ip link set eth1 up
  ip route add default via 192.168.10.1
'
# site B + h2 analogous with 192.168.20.0/24 and 203.0.113.1
docker exec clab-site-tun-inet sh -c '
  ip addr add 198.51.100.2/30 dev eth1; ip link set eth1 up
  ip addr add 203.0.113.2/30 dev eth2; ip link set eth2 up
'

Lab 1 — GRE + static routes

docker exec clab-site-tun-sitea sh -c '
  ip tunnel add gre1 mode gre remote 203.0.113.1 local 198.51.100.1 ttl 64
  ip link set gre1 up mtu 1400
  ip addr add 10.255.255.1/30 dev gre1
  ip route add 192.168.20.0/24 dev gre1
'
docker exec clab-site-tun-siteb sh -c '
  ip tunnel add gre1 mode gre remote 198.51.100.1 local 203.0.113.1 ttl 64
  ip link set gre1 up mtu 1400
  ip addr add 10.255.255.2/30 dev gre1
  ip route add 192.168.10.0/24 dev gre1
'

Predict

Underlay: sitea pings 203.0.113.1
Overlay: h1 pings h2
tcpdump on inet shows GRE (proto 47), not plain inner ICMP as clear tenant IP without decap context

Observe

docker exec clab-site-tun-sitea ping -c 2 203.0.113.1
docker exec clab-site-tun-h1 ping -c 3 192.168.20.10
docker exec clab-site-tun-inet tcpdump -ni eth1 -c 8 proto 47
docker exec clab-site-tun-sitea ip route get 192.168.20.10

Fix

Symptom	Check
No underlay	inet addrs, routes
Underlay OK overlay fail	gre remote/local swap, routes to LAN via gre
One-way	missing return route

Lab 2 — MTU blackhole drill

docker exec clab-site-tun-h1 ping -c 2 -M do -s 1472 192.168.20.10
docker exec clab-site-tun-h1 ping -c 2 -M do -s 1372 192.168.20.10

Predict

Large DF pings fail; smaller succeed if gre mtu 1400.
Fix: lower inner interface MTU, TCP MSS clamp, or raise underlay MTU.

docker exec clab-site-tun-sitea iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Lab 3 — Underlay failure

docker exec clab-site-tun-inet ip link set eth2 down
docker exec clab-site-tun-h1 ping -c 5 192.168.20.10
docker exec clab-site-tun-inet ip link set eth2 up

Predict

Overlay dies with underlay; no magic. Multi-homed underlay would need dual tunnels + routing.

Route-based vs policy-based VPN

Model	Idea	Lab analogue
Route-based	Tunnel is an interface; RIB steers prefixes into it	GRE, WireGuard `wg0`, VTI
Policy-based	SPD selects traffic by ACL selectors for crypto	classic “interesting traffic” IPsec

Route-based composes better with dynamic routing (run OSPF/BGP over tunnel—carefully).

Lab 4 — WireGuard route-based (if packages available)

# Generate keys on both sides (persist in lab folder for real work)
docker exec clab-site-tun-sitea sh -c '
  apk add --no-cache wireguard-tools
  wg genkey | tee /tmp/a.key | wg pubkey > /tmp/a.pub
'
# exchange pubs out of band in lab notes

docker exec clab-site-tun-sitea sh -c '
  ip link add wg0 type wireguard
  ip addr add 10.255.255.1/30 dev wg0
  wg set wg0 private-key /tmp/a.key listen-port 51820 \
    peer B_PUBLIC_KEY endpoint 203.0.113.1:51820 allowed-ips 192.168.20.0/24,10.255.255.2/32
  ip link set wg0 up
  ip route add 192.168.20.0/24 dev wg0
'

Mirror on siteb with allowed-ips 192.168.10.0/24. Open UDP 51820 on path (inet forwards by default).

Predict → observe

docker exec clab-site-tun-h1 ping -c 3 192.168.20.10
docker exec clab-site-tun-inet tcpdump -ni eth1 -c 5 udp port 51820
docker exec clab-site-tun-sitea wg show

If wireguard-tools or kernel module missing in container, treat as optional—GRE still teaches site tunnels. Host-level WireGuard between namespaces is an alternate craft path from lab-craft chapters.

FRR over GRE (dynamic optional)

Enable zebra + ospfd or bgpd on FRR images replacing Alpine sites:

interface gre1
 ip ospf area 0
router ospf
 ospf router-id 1.1.1.1

Predict: LAN prefixes appear via OSPF over tunnel.
Risk: if underlay also runs OSPF without careful design, you can create recursive routing (tunnel endpoint learned via tunnel). Keep underlay and overlay routing domains separate.

Policy-based IPsec awareness

Selectors: src 192.168.10.0/24 dst 192.168.20.0/24
Peer: 203.0.113.1

Tools: strongSwan, LibreSwan—configs are verbose; for this book, understand selectors + SA state:

# illustrative
ip xfrm state
ip xfrm policy

Debug with state/policy tables + outer ESP captures (proto 50).

Security notes (vendor-agnostic)

GRE alone is not confidential
Authenticate/encrypt on untrusted underlays
Filter what enters the tunnel (no free transit for world)
Manage keys as secrets (not committed plaintext in public repos)

verify.sh sketch

#!/usr/bin/env bash
set -euo pipefail
docker exec clab-site-tun-h1 ping -c2 -W1 192.168.20.10
docker exec clab-site-tun-h2 ping -c2 -W1 192.168.10.10
echo OK

Common mistakes

Mistake	Symptom
Tunnel up, no LAN routes	Only gre peer pings
Recursive routing	Flapping tunnel endpoint
MTU ignored	App failures
NAT on underlay without keepalive	Idle VPN dies
Asymmetric interesting traffic	Policy VPN one-way

Summary

Site tunnels extend private sites across a routed underlay
GRE is the clearest teaching tool; WireGuard is a modern route-based crypto option
Separate underlay reachability from overlay routing
Always test MTU and underlay failure
Prefer route-based models when combining with dynamic routing

Next: VXLAN data plane—L2 segments over L3 underlays with VNI encapsulation.