← Interview Prep

BGP — Theory

The path-vector protocol that glues the Internet together: autonomous systems, attributes, best-path selection, policy, scaling, and security.

BGP (Border Gateway Protocol, current version BGP-4, RFC 4271) is the inter-domain routing protocol of the Internet. Unlike IGPs (OSPF/IS-IS) that optimize for shortest cost inside one administrative domain, BGP is built to exchange reachability between domains and to let each operator express policy — who to send traffic to, who to accept it from, and which path to prefer for business rather than purely topological reasons.

BGP is a path-vector protocol: instead of advertising a metric, it advertises the full list of autonomous systems a prefix must transit (the AS_PATH). That vector is both the loop-prevention mechanism (an AS rejects any route already containing its own ASN) and the raw material for policy. It runs over TCP port 179 — reliability, ordering, and flow control are TCP's job, not BGP's.

Overview: autonomous systems and the path vector

An Autonomous System (AS) is a network under a single administrative/routing policy, identified by an ASN. BGP speakers exchange NLRI (Network Layer Reachability Information — the prefixes) tagged with path attributes.

ConceptDetail
ASN sizeOriginally 2-byte (0–65535). Exhausted, so 4-byte ASNs (RFC 6793, 0–4294967295) are now standard. Old speakers see 4-byte ASNs as the placeholder AS23456 (AS_TRANS), with the real value carried in the AS4_PATH/AS4_AGGREGATOR optional-transitive attributes.
Private ASNs64512–65534 (2-byte) and 4200000000–4294967294 (4-byte) — used internally, stripped at the edge with remove-private-as.
TransportUnicast TCP/179; the initiator uses an ephemeral source port. Sessions are typically protected with TTL security (GTSM) and/or TCP-AO/MD5.
Message typesOPEN (negotiate version, ASN, hold-time, capabilities), UPDATE (advertise/withdraw NLRI + attributes), KEEPALIVE, NOTIFICATION (error → tear down).
IncrementalBGP sends the full table once at session start, then only incremental updates (deltas) and periodic keepalives — it does not periodically re-flood the whole table.

eBGP vs iBGP

The same protocol behaves differently depending on whether the neighbor is in a different AS (eBGP) or the same AS (iBGP). This distinction drives almost every BGP design decision.

BehavioreBGP (different AS)iBGP (same AS)
Default TTL1 (peers usually directly connected) — raise with ebgp-multihopSet by IGP reachability; not directly connected typically
AS_PATHLocal ASN is prepended on advertisementNot prepended (stays within one AS)
NEXT_HOPRewritten to the advertising router's interface IPUnchanged — carried across the AS as-is (hence next-hop-self)
Admin distance (Cisco)20 (preferred)200
Re-advertisementFreely re-advertised to other peersNever re-advertised to another iBGP peer (loop prevention)

The iBGP loop-prevention rule and full-mesh requirement

Inside an AS there is no AS_PATH growth, so BGP cannot use it to detect loops among iBGP peers. The safeguard is a split-horizon rule: a route learned from an iBGP peer is never advertised to another iBGP peer. The consequence is that every iBGP speaker must hear routes directly from the originator — i.e. iBGP requires a full mesh of sessions: n(n−1)/2 sessions for n routers. That scales quadratically and is the central iBGP scaling problem.

iBGP full mesh for 5 routers = 5*4/2 = 10 sessions

      R1 ───── R2
      │ \     / │
      │   \ /   │
      │   / \   │
      │ /     \ │
      R4 ───── R3
          \   /
           R5   (all pairs peered)

Two mechanisms remove the full-mesh burden:

SolutionHow it works
Route Reflectors (RR) (RFC 4456)A designated RR is allowed to reflect iBGP routes between clients. Clients peer only with the RR(s), not each other, cutting sessions to O(n). Loop prevention shifts from split-horizon to two new optional attributes: ORIGINATOR_ID (RID of the route's originator — drop if it's your own) and CLUSTER_LIST (list of CLUSTER_IDs the route passed through — drop if your cluster-id appears). Reflection rules: client→RR is reflected to all clients + non-clients; non-client→RR is reflected to clients only.
Confederations (RFC 5065)Split one AS into several sub-ASes (member-ASes) that run eBGP-like sessions between them but appear as a single AS externally. The sub-AS numbers ride in the AS_CONFED_SEQUENCE segment and are stripped at the confederation edge, so the outside world sees only the confederation ID.

Neighbor FSM (finite state machine)

A BGP session progresses through six states. Knowing where it is stuck is the fastest way to diagnose a peering problem.

Idle ─▶ Connect ─▶ OpenSent ─▶ OpenConfirm ─▶ Established
   │       │
   │       └─▶ Active ─▶ (retry Connect)
   └── admin down / error resets here
StateMeaning
IdleRefusing connections; waiting to start / after an error. Route to the peer must exist.
ConnectWaiting for the TCP three-way handshake to complete.
ActiveTCP failed; actively retrying to open the connection. Not a "good" state — it means the TCP session isn't coming up.
OpenSentTCP up; sent our OPEN, waiting for the peer's OPEN.
OpenConfirmOPENs exchanged and validated; waiting for the first KEEPALIVE.
EstablishedSession up; UPDATEs flow. This is the only state where prefixes are exchanged.

Stuck-state cheat sheet

Path attributes

Every UPDATE carries attributes classed by two axes: well-known vs optional (must all routers understand it?) and transitive vs non-transitive (should an AS that doesn't recognize it still pass it on?).

CategoryAttributeRole
Well-known mandatoryORIGINHow the prefix entered BGP: i (IGP/network stmt) < e (EGP, legacy) < ? (incomplete/redistributed). Lower is preferred.
AS_PATHOrdered list of ASNs the route traversed. Loop prevention + a tiebreaker (shorter wins). Segments: AS_SEQUENCE, AS_SET (from aggregation).
NEXT_HOPIP to forward toward this prefix. Must be reachable (recursively via the IGP) or the route is invalid.
Well-known discretionaryLOCAL_PREFPreference within the AS — highest wins. The primary knob to steer outbound traffic; iBGP-only (not sent to eBGP peers).
ATOMIC_AGGREGATEFlags that a less-specific aggregate was chosen over more-specifics (path info may have been lost).
Optional transitiveCOMMUNITYArbitrary 32-bit tags (ASN:value) for grouping routes into policy classes. Well-known: NO_EXPORT, NO_ADVERTISE, NO_EXPORT_SUBCONFED, LOCAL_AS. Also Extended & Large Communities (4-byte-ASN safe).
AGGREGATORASN + RID of the router that performed route aggregation.
Optional non-transitiveMED (Multi-Exit Discriminator)Hint to a neighboring AS about which of several links into you to prefer — lowest wins. Used to influence inbound; not propagated beyond the receiving AS by default.

Two more you should name: WEIGHT is a Cisco-proprietary, router-local attribute (never advertised) — highest wins, evaluated first. Local-only means it only affects the box it's set on.

Best-path selection algorithm

When BGP has multiple paths to the same prefix it runs them through an ordered tiebreaker and installs exactly one as best (unless multipath is enabled). The classic Cisco order — memorize it top to bottom:

#CriterionPrefer
0NEXT_HOP reachable?Ignore any path whose next-hop is unresolvable.
1Weight (Cisco, local)Highest
2LOCAL_PREFHighest
3Locally originatedPrefer routes this router network/aggregate/redistributed over learned ones.
4Shortest AS_PATHFewest ASNs (prepends count; AS_SET = 1)
5Lowest ORIGINIGP (i) < EGP (e) < incomplete (?)
6Lowest MEDCompared only among paths from the same neighbor AS by default
7eBGP over iBGPPrefer externally learned paths
8Lowest IGP metric to NEXT_HOP"Hot-potato" — exit via the closest egress
9Oldest eBGP pathPrefer the longest-established (stability); or use multipath here
10Lowest Router IDLowest BGP RID (or ORIGINATOR_ID if reflected)
11Lowest CLUSTER_LIST lengthShortest reflection path
12Lowest neighbor IPFinal deterministic tiebreak
Mnemonic: "We Love Oranges AS Oranges Mean Pure Refreshment"Weight, Local-pref, Originate, AS-path, Origin, MED, Paths (eBGP/iBGP), RID. The first four settle the vast majority of real decisions.

Policy tools and traffic steering

Policy is applied per-neighbor, inbound (routes I accept) or outbound (routes I advertise). The building blocks:

ToolMatches / does
Prefix-listFilter by prefix + length range (le/ge). The standard way to permit/deny specific NLRI.
AS-path access-listRegex over the AS_PATH (e.g. ^$ = locally originated only, _65001$ = originated by AS65001).
Community-listMatch tags to classify routes into policy groups.
Route-mapThe orchestrator: match (prefix-list/as-path/community) then set (local-pref, MED, community, weight, AS-path prepend, next-hop). Applied in or out per neighbor.

Steering traffic: inbound vs outbound

The key mental model — you control your own outbound easily; influencing inbound is only a hint the other side may ignore:

GoalDirection of trafficPrimary tool
Choose which provider I send traffic outOutboundLOCAL_PREF (set inbound on received routes; highest wins, AS-wide) — the strong, reliable knob.
Influence which link others use to reach meInboundAS-path prepend (make one path look longer), MED (between links to one neighbor AS), more-specific advertisement, or communities the upstream honors. All are hints.
! Prefer ISP-A outbound (higher local-pref set inbound):
route-map FROM_ISP_A permit 10
 set local-preference 200
!
! De-prefer this path inbound by prepending my own ASN 3x:
route-map TO_ISP_B permit 10
 set as-path prepend 65010 65010 65010
!
! Tag routes NO_EXPORT so a peer keeps them local:
route-map TO_PEER permit 10
 set community no-export

Scaling & stability

Security

BGP trusts what it hears, so most incidents are route hijacks (someone originates a prefix that isn't theirs) or route leaks (a route propagated against policy, e.g. a customer re-advertising one provider to another). Defenses layer up:

ControlWhat it does
RPKI / ROAA Route Origin Authorization is a signed record binding a prefix to the ASN allowed to originate it, published in the Resource PKI. Routers do ROV (Route Origin Validation), marking paths valid / invalid / not-found and dropping/deprioritizing invalids.
RPKI-to-Router (RTR)RFC 6810/8210 protocol that feeds validated prefix→origin data from a validating cache to the routers, so the router itself doesn't do crypto.
Max-prefix limitneighbor ... maximum-prefix N tears down or warns on a session that suddenly sends too many routes — blunts leaks and misconfigs (e.g. someone re-advertising the full table).
Bogon / martian filteringReject prefixes that should never appear: RFC1918/private space, default-only leaks, unallocated space, prefixes longer than /24 in v4, your own space arriving from outside.
Prefix & AS-path filteringStrict inbound prefix-lists on customer sessions; AS-path filters to enforce customer/peer/provider relationships (basis of the Gao-Rexford valley-free / "no-leak" rules, and BGP roles / RFC 9234 OTC).
Session integrityTCP-AO or MD5 auth, and GTSM/TTL security (ttl-security hops 1) so off-path attackers can't inject into the TCP session.

Prefix hijack basics: because best-path prefers a more-specific prefix and (all else equal) a shorter AS_PATH, an attacker who originates a /24 covering your /22, or falsely inserts themselves closer in the path, can pull traffic. RPKI stops the naive origin-forgery case; path forgery (announcing a valid origin but a fake adjacency) needs BGPsec (path signing, little-deployed) or ASPA (AS Provider Authorization).

Likely follow-up questions

More company-bank questions (Meta-style)

Related: TCP — Theory & Mechanisms · Life of a Packet.