Fabric with FRR

Updated

September 4, 2026

Fabric with FRR

This chapter turns the Clos model into an FRR-operable fabric kit: config layout, unnumbered eBGP option, verification scripts, and failure drills. Open stack only—Containerlab + FRR + Alpine (mid-2026). Overlay remains optional on leaves.

Learning goals

By the end of this chapter you can:

  • Organize a fabric lab repo for N leaves × M spines
  • Implement OSPF or eBGP underlay consistently on FRR
  • Use maximum-paths and read multipath in RIB/FIB
  • Smoke-test a fabric with a single verify.sh
  • Know where to bolt VXLAN/EVPN on leaves later

Lab repo layout

labs/fabric-frr/
  clos22.clab.yml
  addressing.md
  config/
    s1/frr.conf daemons
    s2/...
    l1/...
    l2/...
  scripts/
    render.sh      # optional template expander
    verify.sh
    fail-spine.sh
  captures/
  README.md

Templating (jinja/make/python) pays off at 4×4; 2×2 can be hand-written.

Design defaults for this book

Choice Default
Size 2 spine × 2 leaf
Underlay OSPF area 0 p2p or eBGP
Loopbacks 10.0.0.x/32
Hosts one per leaf, /24
Overlay off (underlay first)
Images quay.io/frrouting/frr:10.2.1 + alpine:3.20

eBGP unnumbered (modern fabric style)

When link-local + IPv6 RA or RFC5549-style patterns available, unnumbered BGP reduces IP inventory. FRR support has matured—verify on your image:

interface eth1
 ip address 10.1.1.1/30
!
router bgp 65101
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor fabric capability extended-nexthop
 neighbor eth1 interface peer-group fabric
 !
 address-family ipv4 unicast
  neighbor fabric activate
  network 10.0.0.11/32
 exit-address-family

If interface peering is painful in your build, use explicit /30 neighbors (Clos chapter)—same topology math.

Numbered eBGP fabric (reliable teaching)

Node ASN
s1 65100
s2 65100
l1 65101
l2 65102

Spines same ASN (or not—both designs exist). Leaves unique.

Spine s1 excerpt

hostname s1
!
interface lo
 ip address 10.0.0.1/32
!
interface eth1
 ip address 10.1.1.2/30
!
interface eth2
 ip address 10.1.3.2/30
!
router bgp 65100
 bgp router-id 10.0.0.1
 neighbor 10.1.1.1 remote-as 65101
 neighbor 10.1.3.1 remote-as 65102
 !
 address-family ipv4 unicast
  network 10.0.0.1/32
  neighbor 10.1.1.1 activate
  neighbor 10.1.3.1 activate
  maximum-paths 4
 exit-address-family
!
line vty

Leaf l1 excerpt

hostname l1
!
interface lo
 ip address 10.0.0.11/32
!
interface eth1
 ip address 10.1.1.1/30
!
interface eth2
 ip address 10.1.2.1/30
!
interface eth3
 ip address 192.168.1.1/24
!
router bgp 65101
 bgp router-id 10.0.0.11
 neighbor 10.1.1.2 remote-as 65100
 neighbor 10.1.2.2 remote-as 65100
 !
 address-family ipv4 unicast
  network 10.0.0.11/32
  network 192.168.1.0/24
  neighbor 10.1.1.2 activate
  neighbor 10.1.2.2 activate
  maximum-paths 4
 exit-address-family
!
line vty

daemons: zebra=yes, bgpd=yes.

OSPF alternate (same wiring)

Reuse Clos chapter OSPF configs; add:

router ospf
 maximum-paths 4

Containerlab topology

Same as Clos chapter clos22—keep one file, swap configs via binds.

Operational show kit

# sessions / neighbors
docker exec clab-clos22-l1 vtysh -c 'show bgp summary'
docker exec clab-clos22-l1 vtysh -c 'show ip ospf neighbor'

# multipath
docker exec clab-clos22-l1 vtysh -c 'show ip route 192.168.2.0/24'
docker exec clab-clos22-l1 ip route show 192.168.2.0/24

# loopback mesh
for d in 10.0.0.1 10.0.0.2 10.0.0.11 10.0.0.12; do
  docker exec clab-clos22-l1 ping -c1 -W1 $d || echo fail $d
done

# hosts
docker exec clab-clos22-h1 ping -c2 192.168.2.10

verify.sh

#!/usr/bin/env bash
set -euo pipefail
fail() { echo "FAIL: $*" >&2; exit 1; }
P=clab-clos22

docker exec $P-l1 ping -c1 -W1 10.0.0.12 || fail "l1->l2 lo"
docker exec $P-l1 ping -c1 -W1 10.0.0.1 || fail "l1->s1 lo"
docker exec $P-h1 ping -c2 -W1 192.168.2.10 || fail "h1->h2"
docker exec $P-h2 ping -c2 -W1 192.168.1.10 || fail "h2->h1"

# multipath soft check: at least one route line with multiple nexthops or two entries
R=$(docker exec $P-l1 vtysh -c 'show ip route 192.168.2.0/24' || true)
echo "$R" | grep -Eq '192.168.2.0|via' || fail "no route to h2 subnet"

echo OK-FABRIC

Failure scripts

#!/usr/bin/env bash
# fail-spine.sh s1
set -euo pipefail
SP=${1:-s1}
docker exec clab-clos22-$SP ip link set eth1 down
docker exec clab-clos22-$SP ip link set eth2 down
echo "$SP uplinks down; run ping/traceroute; then restore"
#!/usr/bin/env bash
# restore-spine.sh s1
SP=${1:-s1}
docker exec clab-clos22-$SP ip link set eth1 up
docker exec clab-clos22-$SP ip link set eth2 up

Predict → observe → fix

Drill Predict Observe
Spine down Paths via other spine single nexthop
Leaf uplink down Still works 1 BGP/OSPF neighbor
Wrong ASN Session down show bgp summary Active/Idle
Missing network stmt Host subnet absent RIB missing

Policy on fabric (light)

Even underlays need hygiene:

# do not accept default from random peers in fabric
# do not advertise lab management prefixes
ip prefix-list FABRIC-LO seq 5 permit 10.0.0.0/24 le 32
route-map EXPORT-FABRIC permit 10
 match ip address prefix-list FABRIC-LO

Apply out toward spines if you practice policy discipline.

Adding overlay later (hook points)

On each leaf:

  1. Keep underlay BGP/OSPF as-is
  2. Add VTEP on loopback
  3. EVPN AF on BGP or static VXLAN to other leaf loopbacks
  4. Spines remain ignorant of VNIs
leaf = underlay router + VTEP + (optional) IRB
spine = underlay router only

Scale checklist 2×2 → 4×4

Item Action
IP plan Spreadsheet / addressing.md generator
Configs Templates
RAM FRR light; watch host memory
verify Loop all leaf pairs
Cabling matrix Automate clab links generation

Observability

  • Enable FRR logging to bound files
  • Graph: containerlab graph -t clos22.clab.yml
  • Optional: scrape with your ops chapter tools

Optional SONiC / SR Linux leaf

Replace one leaf image with community NOS; keep underlay protocol same. Document CLI mapping for show bgp / show ip route. Goal: multi-implementation confidence (next chapter deepens this).

Common mistakes

Mistake Symptom
Spines advertising host defaults weirdly Traffic trombones
maximum-paths left at 1 No ECMP
Overlay on spine Operational regret
No cold deploy test Config drift in running only
Same router-id BGP/OSPF chaos

Summary

  • FRR is enough to run a real teaching fabric underlay
  • Standardize repo layout, ASNs, and verify scripts
  • Prove ECMP and spine loss before overlays
  • Leaves are the VTEP attachment point later
  • Policy-light underlays still need prefix discipline

Next: multi-implementation lab—FRR + Linux + optional SR Linux in one topology.