BGP Protocol Architecture

Author

K19G

Updated

October 31, 2026

BGP Protocol Architecture

The Border Gateway Protocol version 4 (BGP-4), defined in RFC 4271, is an application-layer protocol running over TCP. Its protocol mechanics dictate how peering sessions are negotiated, maintained, and torn down, and establish the structural framework that modern routing security extensions must operate within.

The BGP Message Types: OPEN, UPDATE, NOTIFICATION, KEEPALIVE

All BGP messages share a standard 19-byte header:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+                                                               +
|                                                               |
+                               Marker                          +
|                              (16 octets)                      |
+                                                               +
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Length (2 octets)    |   Type (1 oct)  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  • Marker (16 octets): Set to all ones for synchronization and legacy authentication compatibility.
  • Length (2 octets): Total message length in bytes (minimum 19, standard maximum 4096 bytes; extended to 65,535 bytes under RFC 8654 Extended Message support).
  • Type (1 octet): Indicates the payload message type:
Type Name Purpose
1 OPEN Negotiates session parameters (BGP version, ASN, Hold Time, BGP Identifier, Capabilities TLVs).
2 UPDATE Advertises reachable routes with path attributes, and withdraws unreachable routes.
3 NOTIFICATION Signals a fatal protocol error or graceful session shutdown; terminates the peering session immediately.
4 KEEPALIVE Periodic heartbeat message (19-byte header only) to verify neighbor liveness before Hold Time expires.
5 ROUTE-REFRESH (RFC 2918) Dynamically requests a peer to re-send its Adj-RIB-Out without resetting the TCP session.

The BGP Finite State Machine: From Idle to Established

The BGP Finite State Machine (FSM) defines the lifecycle of a peering session across six deterministic states:

           +--------------------------------------------------+
           |                                                  |
           v                                                  |
       +-------+  Start Event                                 |
       | Idle  | --------------------+                        |
       +-------+                     |                        |
           ^                         v                        |
           |                  +--------------+                |
           |                  | Connect      |                |
           |                  +--------------+                |
           |                     |          |                 |
           |      TCP Syn Fail   |          | TCP Connected   |
           |                     v          v                 |
           |               +----------+  +----------+         |
           |               | Active   |  | OpenSent |         |
           |               +----------+  +----------+         |
           |                     |              |             |
           |        TCP Success  |              | Recv OPEN   |
           |                     v              v             |
           |                  +--------------------+          |
           |                  | OpenConfirm        |          |
           |                  +--------------------+          |
           |                            |                     |
           |                            | Recv KEEPALIVE      |
           |                            v                     |
           |                  +--------------------+          |
           |                  | Established        | ---------+
           |                  +--------------------+  Error / NOTIFICATION
           +------------------------------------------+
  1. Idle: The initial state. The router allocates resources, initializes variables, and waits for a start event (manual or automatic). In response, it initiates a TCP connection and transitions to Connect.
  2. Connect: The router attempts a 3-way TCP handshake with the peer on port 179.
    • If TCP succeeds: it transmits an OPEN message and transitions to OpenSent.
    • If TCP fails: it starts the ConnectRetry timer and transitions to Active.
  3. Active: The router actively listens for incoming TCP connections while attempting to re-establish the connection. If the timer expires without success, it reverts to Connect.
  4. OpenSent: The router has sent its OPEN message and awaits an OPEN message from the peer. Upon receipt, it validates the peer’s ASN, version, and capabilities:
    • If valid: sends a KEEPALIVE and moves to OpenConfirm.
    • If invalid: sends a NOTIFICATION with error code and resets to Idle.
  5. OpenConfirm: Awaiting a KEEPALIVE or NOTIFICATION from the peer. Receiving a KEEPALIVE confirms mutual parameter acceptance, transitioning the session to Established.
  6. Established: The operational state. Peers exchange reachability updates (UPDATE), route refresh requests, and periodic heartbeats (KEEPALIVE). Any protocol violation or expired hold timer triggers a NOTIFICATION and tears the session back to Idle.

TCP Port 179 and Transport Considerations

By delegating transport-layer responsibilities to TCP (Port 179), BGP inherited several key characteristics and vulnerabilities:

  • Reliability and Flow Control: BGP avoids implementing retransmission, fragmentation, windowing, or congestion control at the routing layer. TCP guarantees byte-stream delivery.
  • SYN-Flood and RST Attacks: Because TCP sessions are vulnerable to blind RST injection and SYN-flood starvation, BGP requires protection mechanisms:
    • TCP MD5 Signature Option (RFC 2385): Embeds an MD5 digest computed over the TCP header, payload, and shared secret in every TCP segment. While cryptographically obsolete (MD5 collisions), it remains widely deployed across legacy hardware.
    • TCP Authentication Option (TCP-AO, RFC 5925): Modern replacement for MD5, utilizing HMAC-SHA-1 or AES-128-CMAC with cryptographic key rollover capabilities.
    • Generalized TTL Security Mechanism (GTSM, RFC 5082): Exploits packet hop counts to block off-path attackers. For directly connected eBGP peers, the outgoing IP TTL is set to \(255\). The receiver verifies that incoming packets have a TTL of \(255\) (or \(254\)). An off-path attacker across the Internet cannot craft a packet that arrives with a TTL of \(255\).

BGP-4 UPDATE Message Structure and TLV Encoding

The UPDATE message is the operational workhorse of BGP:

+-----------------------------------------------------+
|   Withdrawn Routes Length (2 octets)                |
+-----------------------------------------------------+
|   Withdrawn Routes (variable length)                |
|   [Prefix Length (1 octet), Prefix (variable)]      |
+-----------------------------------------------------+
|   Total Path Attribute Length (2 octets)            |
+-----------------------------------------------------+
|   Path Attributes (variable length)                 |
|   [Flags (1 oct), Type (1 oct), Length (1-2), Value]|
+-----------------------------------------------------+
|   Network Layer Reachability Information (NLRI)     |
|   [Length (1 octet), Prefix (variable)]             |
+-----------------------------------------------------+

Each path attribute is encoded in a Type-Length-Value (TLV) triple preceded by an Attribute Flags byte: - Bit 0 (Optional): Set to \(0\) if Well-Known; \(1\) if Optional. - Bit 1 (Transitive): Set to \(1\) if Transitive; \(0\) if Non-Transitive. (All Well-Known attributes are transitive). - Bit 2 (Partial): Set to \(1\) if an upstream router forwarded an unrecognized optional transitive attribute without understanding it. - Bit 3 (Extended Length): Set to \(0\) if Attribute Length is 1 octet; set to \(1\) if Attribute Length is 2 octets.

Capabilities Negotiation and Modern Extensions

During the initial OPEN message exchange, BGP speakers negotiate optional capabilities (RFC 5492) using Optional Parameters TLVs:

  • Multiprotocol BGP (MP-BGP, RFC 4760): Extends BGP beyond IPv4 unicast by defining Address Family Identifiers (AFI: IPv4=1, IPv6=2) and Subsequent AFIs (SAFI: Unicast=1, Multicast=2, EVPN=70). This decouples protocol transport from payload reachability.
  • 4-Byte ASN Support (RFC 6793): Allows routers to negotiate 32-bit AS numbers, preserving backwards compatibility with legacy 2-byte speakers through the pseudo-ASN AS_TRANS (23456).
  • Graceful Restart (RFC 4724): Preserves data-plane forwarding across control-plane restarts, preventing temporary routing flaps during software upgrades.
  • Enhanced Route Refresh (RFC 7313): Guarantees synchronization between peers during policy reloads.