ICMP & Path MTU

Updated

September 4, 2026

ICMP & Path MTU

Small pings succeed while large transfers hang: classic MTU / PMTUD failure. This chapter treats ICMP as a control helper and shows you how to find black holes with deliberate packet sizes.

Learning goals

By the end of this chapter you can:

  • Explain key ICMP types used in operations
  • Test reachability with controlled sizes and DF bit
  • Predict PMTUD behavior when ICMP “fragmentation needed” is filtered
  • Clamp MTU on a lab link and measure the symptom
  • Document a harden checklist for MTU in topologies

ICMP essentials (IPv4)

Type Role
Echo request/reply ping
Destination unreachable no route, port unreachable, frag needed
Time exceeded TTL expired (traceroute)
Redirect host guidance (often ignored/noisy) 
ping -c 3 10.0.0.1
ping -c 3 -s 1400 10.0.0.1
ping -M do -s 1472 10.0.0.1   # DF set; 1472 + 28 = 1500

-M do (Linux) sets Don’t Fragment. Adjust payload so total IP packet hits MTU boundary.

IPv6 uses ICMPv6 extensively (ND, PMTUD); similar lab methods with ping -6.

Path MTU discovery (idea)

  1. Sender assumes path MTU (often interface MTU)
  2. Sends DF packets
  3. Hop with smaller MTU sends ICMP frag needed + next-hop MTU
  4. Sender caches smaller PMTU

If step 3 is filtered, large packets blackhole; small pings work.

Lab: MTU mismatch

h1 -- r1 ======== r2 -- h2
         MTU 1400 on r1-r2 link

Topology

name: mtu-lab

topology:
  nodes:
    r1:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils iptables
        - sysctl -w net.ipv4.ip_forward=1
        - ip addr add 192.168.1.1/24 dev eth1
        - ip addr add 10.0.0.1/24 dev eth2
        - ip link set eth1 up
        - ip link set eth2 up
        - ip link set eth2 mtu 1400
        - ip route add 192.168.2.0/24 via 10.0.0.2
    r2:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils iptables
        - sysctl -w net.ipv4.ip_forward=1
        - ip addr add 192.168.2.1/24 dev eth1
        - ip addr add 10.0.0.2/24 dev eth2
        - ip link set eth1 up
        - ip link set eth2 up
        - ip link set eth2 mtu 1400
        - ip route add 192.168.1.0/24 via 10.0.0.1
    h1:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils
        - ip addr add 192.168.1.10/24 dev eth1
        - ip link set eth1 up
        - ip route add default via 192.168.1.1
    h2:
      kind: linux
      image: alpine:3.20
      exec:
        - apk add --no-cache iproute2 iputils
        - ip addr add 192.168.2.10/24 dev eth1
        - ip link set eth1 up
        - ip route add default via 192.168.2.1

  links:
    - endpoints: ["h1:eth1", "r1:eth1"]
    - endpoints: ["h2:eth1", "r2:eth1"]
    - endpoints: ["r1:eth2", "r2:eth2"]

Predict

  • ping -c 2 192.168.2.10 small default works
  • Large DF ping across path may fail or trigger PMTUD depending on sizes
  • Capture may show ICMP unreachable frag needed if not filtered

Observe

docker exec clab-mtu-lab-h1 ping -c 2 192.168.2.10
docker exec clab-mtu-lab-h1 ping -M do -s 1472 -c 2 192.168.2.10
docker exec clab-mtu-lab-h1 ping -M do -s 1200 -c 2 192.168.2.10
docker exec clab-mtu-lab-r1 ip link show eth2

Inject: drop ICMP unreachable on r1

docker exec clab-mtu-lab-r1 iptables -A FORWARD -p icmp --icmp-type fragmentation-needed -j DROP
# also output path variants may need INPUT/OUTPUT depending on generation point
docker exec clab-mtu-lab-r1 iptables -A OUTPUT -p icmp --icmp-type fragmentation-needed -j DROP

Retest large DF pings / large TCP if iperf available.

Predict: hard blackhole for large DF packets.

Restore

docker exec clab-mtu-lab-r1 iptables -F

Harden

  • Match MTUs both ends of a link
  • Do not filter all ICMP on transit routers
  • For TCP, MSS clamping on edges sometimes used as workaround—prefer fixing MTU/ICMP
# example TCP MSS clamp (awareness; use carefully)
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1360

Traceroute and ICMP filters

traceroute -n 192.168.2.10

If time-exceeded filtered, traceroute incomplete while data works. Journal both ICMP classes separately: PMTUD vs traceroute.

Capture workflow

docker exec clab-mtu-lab-r1 tcpdump -ni eth2 -c 40 icmp or '(ip[6:2] & 0x1fff) != 0'
docker exec clab-mtu-lab-h1 ping -M do -s 1472 -c 3 192.168.2.10

Look for:

  • Echo requests size
  • ICMP type 3 code 4 (frag needed)
  • Whether sender reduces size after

iperf3 large transfers

# on h2
apk add iperf3
iperf3 -s
# on h1
iperf3 -c 192.168.2.10

Compare before/after MTU clamp and ICMP drop. TCP may stall if PMTUD blackhole.

IPv6 notes

IPv6 routers do not fragment in the same way; PMTUD is even more critical. Minimum link MTU is 1280.

ping -6 -M do -s 1400 <dst>
ip -6 route get <dst>

Operational checklist

Check Command / action
Interface MTU ip link
End-to-end small ping ping -c 3
DF large ping ping -M do -s ...
ICMP allowed policy review
TCP test iperf3 / curl large
Tunnel overhead account for encapsulation

Tunnels (later chapters) reduce effective MTU—plan overhead.

Predict worksheet

Path MTU 1500 except one hop 1400; ICMP filtered.

  1. Ping default size 56 payload — ?
  2. DF packet size 1500 — ?
  3. TCP bulk transfer — ?

Answers: (1) usually ok; (2) blackhole; (3) often hang or extreme slowness depending on stack workarounds.

verify.sh additions

#!/usr/bin/env bash
set -euo pipefail
docker exec clab-mtu-lab-h1 ping -c 2 -W 1 192.168.2.10
# expect success for moderate DF size under 1400 path
docker exec clab-mtu-lab-h1 ping -M do -s 1200 -c 2 -W 1 192.168.2.10
echo OK

Optional: assert large 1472 DF fails when MTU 1400—can be a negative test in failure mode only.

Summary

  • ICMP is part of healthy networks (PMTUD, traceroute), not pure “attack surface” to zero
  • Size-controlled pings expose MTU issues small pings hide
  • Filtering frag-needed creates blackholes
  • Match link MTUs; account for encapsulation overhead
  • Always pair ping with a bulk TCP test when performance is in question

Next part: interior routing—OSPF and BGP basics plus policy/filters on free FRR images.