← Interview Prep

EVPN — Life of a Packet

Tracing a frame through a VXLAN-EVPN fabric in five cases: L2 simple, L2 all-active multihoming (aliasing/DF/split-horizon), L3 simple (symmetric IRB), L3 multihoming, and multicast/BUM (ingress replication vs underlay multicast) — hop by hop with the route types involved.

Theory sticks once you trace a real packet. This walks a frame through a VXLAN-EVPN fabric in the five cases you'll be asked to draw: L2 simple, L2 multihoming, L3 simple, L3 multihoming, and multicast/BUM. Background: EVPN — Theory and VLAN & VXLAN.

The fabric: leaves are VTEPs (L1, L2, L3…), spines are a pure IP underlay (they only route the outer header). L2VNI = a bridge domain, L3VNI = a VRF. Every leaf owns the same anycast gateway IP+MAC. MACs/IPs are learned in BGP EVPN before traffic flows.

The encapsulation a VTEP adds is always the same shape:

[ outer Eth | outer IP (VTEP→VTEP) | UDP dst 4789 | VXLAN (VNI) | inner frame ]
  the UDP source port = hash(inner flow)  → the spines ECMP on it

Control plane — how the routes & L2 tables get signalled (before any packet)

Nothing above forwards until BGP EVPN has populated each leaf's tables. The underlay comes up (IGP/eBGP), the EVPN overlay peers (often iBGP via RR on the spines), and then the route types build state:

Route type	Populates (the table it builds)
Type 2 (MAC/IP)	The MAC-VRF / L2 table: remote MAC → {remote VTEP IP, L2VNI}. Its IP field also fills the ARP/ND suppression table (MAC↔IP), and — with the L3VNI + Router-MAC — a /32 host route in the IP-VRF.
Type 5 (IP-prefix)	The IP-VRF routing table: prefix → {remote VTEP, L3VNI, egress Router-MAC} — summaries, external, silent hosts.
Type 1 per-EVI (A-D)	Aliasing: marks the ESI reachable via several VTEPs → an ECMP set for that segment.
Type 1 per-ES (A-D)	The ESI/split-horizon label and the handle for mass withdrawal (one route to pull all MACs on a failed link).
Type 4 (ES)	Discovers the PEs on a shared ES (ES-Import RT) and runs DF election per EVI.
Type 3 (IMET)	The BUM flood list per VNI — the ingress-replication set (or the underlay multicast group).

So each leaf ends up with: a MAC table (VTEP next-hops), an ARP/ND table, an IP-VRF (host + prefix routes with L3VNI/Router-MAC), an ECMP/DF/split-horizon view of every ESI, and a per-VNI flood list. Local MACs are learned from the wire and re-advertised as Type 2. On a link/host change, a withdrawal updates these tables — that's the whole convergence story.

1 · L2 simple (same subnet, single-homed)

Host A on Leaf1 → Host B on Leaf2, same subnet (same L2VNI 10010).

1. A resolves B's MAC by ARP — Leaf1 answers locally from its EVPN table (ARP suppression), since B's MAC+IP arrived earlier in a Type 2 route. No fabric flood.
2. A sends the frame, Dst = B-MAC. Leaf1 looks up B-MAC → Type 2 says behind Leaf2's VTEP, VNI 10010.
3. Leaf1 VXLAN-encaps: outer IP Leaf1→Leaf2, VNI 10010, inner = A→B frame unchanged.
4. Spines route on the outer IP (ECMP on the UDP source port); they never see the VNI or inner frame.
5. Leaf2 decaps, VNI 10010 → bridge domain, forwards out B's port. Pure bridging, no routing.

2 · L2 multihoming (all-active ESI)

Same subnet, but B is dual-homed to Leaf2 and Leaf3 via one LACP bundle (a shared ESI).

1. Leaf1 learned B-MAC via Type 2 (say Leaf2 advertised it) and learned from the Type 1 per-EVI A-D that the ESI is reachable via both Leaf2 and Leaf3 — aliasing.
2. Leaf1 installs an ECMP next-hop {Leaf2, Leaf3} and hashes A→B's flow → encaps to (say) Leaf3, even though only Leaf2 advertised the MAC.
3. Leaf3 decaps and forwards to B on its local ESI link.
4. BUM toward B is sent only by the DF; split-horizon (local-bias) stops a frame echoed to the peer VTEP from re-entering B.
5. Leaf2's link to B fails → Leaf2 sends one Type 1 per-ES withdrawal (mass withdrawal) → Leaf1 drops Leaf2 from the ECMP set and sends everything to Leaf3. Sub-second, no per-MAC relearn.

3 · L3 simple (inter-subnet, single-homed, symmetric IRB)

Host A (subnet X, VNI 10010, Leaf1) → Host C (subnet Y, VNI 10020, Leaf2). Different subnets → routing.

1. A sends to its default gateway MAC — which is the anycast gateway present locally on Leaf1. So Leaf1 routes (the distributed gateway; no hairpin to a central router).
2. Leaf1 looks up C in the VRF. C's /32 came via Type 2 (MAC+IP) from Leaf2, carrying the L3VNI 50000 and Leaf2's Router-MAC.
3. Symmetric IRB: Leaf1 rewrites inner Dst-MAC = Leaf2's Router-MAC, encaps VXLAN VNI = L3VNI 50000, outer IP Leaf1→Leaf2.
4. Leaf2 decaps, VNI 50000 → the VRF, routes to C's subnet, rewrites Dst-MAC = C-MAC, sends out C's port. Both leaves routed (route-route) via the common L3VNI.

For a subnet/summary rather than a host, the same flow uses a Type 5 IP-prefix route instead of Type 2.

4 · L3 multihoming (inter-subnet, destination dual-homed)

As above, but C is dual-homed to Leaf2 + Leaf3 (ESI), different subnet from A.

1. Leaf1 routes into the L3VNI; C's host route is reachable via Leaf2 and Leaf3 (advertised by both, or via Type 1 aliasing) → ECMP in the VRF {Leaf2, Leaf3}.
2. Leaf1 hashes the flow, sets inner Dst-MAC = the chosen leaf's Router-MAC, VNI = L3VNI, and encaps to it.
3. That leaf decaps → VRF → routes to C locally.
4. Failure of one leaf's ESI link → mass withdrawal / route withdraw → Leaf1 reconverges to the survivor.

5 · Multicast / BUM

A sends broadcast, unknown-unicast, or multicast in VNI 10010 — it must reach every VTEP with a receiver in that VNI.

1. Each remote leaf decaps and floods within VNI 10010 to its local members.
2. For a multihomed receiver, only the DF forwards the BUM copy down the ESI, and split-horizon prevents the non-DF peer from duplicating it.
3. Most "BUM" is avoided entirely: ARP is answered locally (Type 2 suppression), and unicast MACs are already known — so true flooding is rare.