Chapter 12: Network Performance Tuning in Practice

Updated

October 31, 2026

Chapter 12: Network Performance Tuning in Practice

Theoretical knowledge of the Linux network stack must be translated into concrete, reproducible configuration settings for production hosts. Misconfigured network parameters can cap a 100Gbps network link to 1Gbps, exhaust ephemeral ports, trigger bufferbloat, or crash a server under connection spikes.

This chapter details practical network tuning recipes, formulas for socket buffer sizing, congestion control selection, hardware multiqueue configuration, and validation methodologies.

Core TCP Parameters and Their Effects

The following kernel parameters control connection lifecycle, backlog queues, and memory consumption:

# Increase backlog queues for incoming SYN packets and established connections
sudo sysctl -w net.core.somaxconn=4096            # Max socket listen backlog
sudo sysctl -w net.ipv4.tcp_max_syn_backlog=8192  # Max half-open SYN connection queue

# Reuse sockets in TIME_WAIT state for new outgoing connections (safe for clients)
sudo sysctl -w net.ipv4.tcp_tw_reuse=1

# Reduce TIME_WAIT timeout from 60s to 30s to free socket structures faster
sudo sysctl -w net.ipv4.tcp_fin_timeout=30

# Expand ephemeral port range for high-concurrency client/proxy hosts
sudo sysctl -w net.ipv4.ip_local_port_range="10240 65535"

# Disable TCP slow start after idle to prevent latency penalties on idle connections
sudo sysctl -w net.ipv4.tcp_slow_start_after_idle=0

Socket Buffer Sizing Strategy

To maximize single-stream TCP throughput over long distances, socket buffers must be sized to accommodate the Bandwidth-Delay Product (BDP):

\[\text{BDP} = \text{Bandwidth (bits/sec)} \times \text{Round-Trip Time (RTT in seconds)}\]

Practical Sizing Calculation

Suppose a 10Gbps link connects two data centers with an RTT of 40ms (\(0.040\text{ seconds}\)): \[\text{BDP} = 10 \times 10^9 \text{ bps} \times 0.040 \text{ s} = 400,000,000 \text{ bits} = 50,000,000 \text{ bytes} \approx 50 \text{ MB}\]

If the maximum receive buffer (tcp_rmem max) is left at the typical default of 6MB, the TCP window will cap out at 6MB, limiting the maximum achievable throughput to: \[\text{Max Throughput} = \frac{6 \text{ MB}}{0.040 \text{ s}} = 150 \text{ MB/s} = 1.2 \text{ Gbps}\]

The remaining 8.8 Gbps of link capacity will sit completely unused!

# Production tuning for high-BDP 10G/40G links
sudo sysctl -w net.core.rmem_max=67108864          # 64 MB
sudo sysctl -w net.core.wmem_max=67108864          # 64 MB
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 67108864"
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 67108864"

[!CAUTION] On reverse-proxy servers handling 500,000 concurrent client connections, setting tcp_rmem max to 64MB risks instant kernel Out-Of-Memory (OOM) crashes if buffer usage expands. High-concurrency edge nodes require smaller conservative buffer limits (e.g., 2MB to 4MB max).

Congestion Control Algorithm Selection

Modern Linux supports dynamically loading congestion control modules:

# Enable BBR congestion control
sudo modprobe tcp_bbr
echo "tcp_bbr" | sudo tee -a /etc/modules-load.d/bbr.conf
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

# Verify active algorithm
sysctl net.ipv4.tcp_congestion_control
  • Use BBR: For WAN services, internet-facing APIs, cross-region replication, and lossy networks. BBR prevents packet loss from throttling bandwidth and maintains minimal queueing delay.
  • Use CUBIC: For ultra-low-latency intra-datacenter networks where packet loss is near zero and packet queueing in switch buffers is tightly bounded.

Multiqueue Configuration and RSS Tuning

High-speed NICs require symmetric allocation of queues across physical CPU cores:

# Check maximum supported vs currently configured hardware queues
ethtool -l eth0

# Set hardware queues to match physical core count (e.g., 16 queues)
sudo ethtool -L eth0 combined 16

# Increase NIC ring buffer sizes to absorb traffic bursts without packet drops
ethtool -g eth0
sudo ethtool -G eth0 rx 4096 tx 4096

Pinning IRQs with irqbalance or Static Scripts

Ensure that network interrupts are pinned to the CPU cores belonging to the same NUMA node as the PCIe slot hosting the NIC:

# Verify NUMA node of network interface
cat /sys/class/net/eth0/device/numa_node

# View current IRQ affinities for NIC queues
grep eth0 /proc/interrupts

Validating Network Performance Changes

Never assume a network tuning parameter succeeded without rigorous verification:

# Test raw bandwidth and throughput using iperf3
# Server:
iperf3 -s -p 5201

# Client (multiple parallel streams to saturate link):
iperf3 -c 10.0.0.1 -p 5201 -P 8 -t 30

# Inspect TCP retransmissions, drops, and buffer errors during the test
nstat -z | grep -E "TcpRetransSegs|TcpOutRsts|TcpInErrors"

# Check for NIC-level packet drops and buffer overflows
ethtool -S eth0 | grep -E "drop|miss|error|fifo"

If TcpRetransSegs or rx_dropped increment during the benchmark, the system is suffering from buffer starvation, interrupt drops, or switch-level congestion.