Interconnecting data centers across a WAN (DCI): the L2-stretch vs L3-routed decision, EVPN as the unifying control plane, active-active steering, and the failure-domain traps.
Once you have more than one data center you have to answer a single architectural question: do you route between them (L3) or do you stretch a subnet across them (L2)? Everything else in Data Center Interconnect (DCI) — the transport, the control plane, the gateway model, the traffic-steering design — falls out of that decision. The interview version of this topic is almost always "you have two leaf-spine fabrics; connect them — walk me through the options and the tradeoffs."
The one-line rule: prefer L3 interconnect and keep each DC its own failure domain; use L2 extension only where the application genuinely forces it (subnet stretch, live VM mobility, clustering) — and when you do, treat it as extending a broadcast/failure domain across the WAN and engineer the blast-radius controls up front.
The business drivers determine how much coupling you actually need — and coupling is what costs you resilience. Keep the driver in mind, because it decides L2 vs L3:
| Driver | What it needs | L2 or L3? |
|---|---|---|
| Geo-redundancy / DR | A standby site that can take over; recovery point/time objectives (RPO/RTO) | Usually L3 (re-address or anycast on failover) |
| Active-active | Both sites serving the same service simultaneously | L3 + anycast preferred; L2 only if the app demands a shared subnet |
| Workload / VM mobility | Move a running VM keeping its IP (vMotion, live migration) | L2 stretch (same subnet both sides) |
| Capacity / bursting | Spill compute into a second site | L3 (new prefixes advertised from the new site) |
| Backup / replication | Bulk storage replication, snapshots, DB sync | L3 (high-throughput routed path) |
| Legacy clustering / heartbeat | Clusters that require L2 adjacency or shared VLAN | L2 stretch (the classic forcing function) |
Notice that only a minority of drivers truly require L2. That is the point interviewers want you to make: stretching L2 is a liability you accept for a specific application requirement, not a default.
Two leaf-spine (Clos) fabrics, each with a pair of border leaves (a.k.a. DC gateways) that face the WAN. The border leaf is where the DC overlay meets the WAN transport — the control-plane and data-plane stitching point.
DATA CENTER 1 WAN CORE DATA CENTER 2
┌──────────────────────────────┐ ┌──────────────────────────────┐
│ spine1 spine2 │ │ spine1 spine2 │
│ ╱ ╲ ╱ ╲ │ │ ╱ ╲ ╱ ╲ │
│ ╱ ╲ ╱ ╲ │ │ ╱ ╲ ╱ ╲ │
│ leaf leaf leaf leaf │ │ leaf leaf leaf leaf │
│ │ │ │ │ │ │ │ │ │ │ │
│ [srv] [srv] [srv] [srv] │ │ [srv] [srv] [srv] [srv] │
│ border-leaf pair │ │ border-leaf pair │
│ (DC-GW1a/1b) │ │ (DC-GW2a/2b) │
└───────────────┬──────────────┘ └───────────────┬──────────────┘
│ │
│ EVPN peering (eBGP/MP-BGP) MPLS-SR / SR-TE / DWDM │
└────────────▶ PE ══════════ P ══════════ P ══════════ PE ◀────────┘
WAN provider / self-built SR core
VXLAN or MPLS tunnels ride end-to-end over the WAN underlayTwo families: keep the DCs as separate L3 domains, or bridge a broadcast domain between them. This is the table to be able to draw from memory.
| Option | How | Pros | Cons / risk |
|---|---|---|---|
| (1) L3 interconnect recommended default |
Routed handoff between fabrics — EVPN Type-5 (IP-prefix routes) or L3VPN (VPNv4/v6). Each DC keeps its own subnets; only prefixes/host-routes cross the WAN. | Cleanest; contained failure domain (no broadcast across the WAN); scales; easy anycast; failures stay local. | No subnet stretch — can't do IP-preserving VM mobility or L2 clustering. Apps must tolerate re-addressing or use anycast/DNS. |
| (2) L2 extension only when forced |
EVPN-VXLAN (Type-2 MAC/IP + Type-3 IMET) over the WAN, EVPN-MPLS, legacy VPLS, or dark-fiber / DWDM pseudo-wire. Same subnet on both sides. | Enables subnet stretch, live VM mobility, L2 clustering, IP-preserving migration. | Dangerous: extends the broadcast & failure domain across the WAN — BUM flooding, L2 loops, and broadcast storms now span sites. A loop or storm in DC1 can take down DC2. |
If you must stretch L2, the whole game is shrinking the blast radius. Interviewers expect you to list these unprompted:
| Mitigation | What it controls |
|---|---|
| BUM control | Handle Broadcast/Unknown-unicast/Multicast explicitly — ingress replication or underlay multicast, and rate-limit it so a storm can't saturate the WAN. |
| Storm control | Per-port/per-VLAN broadcast/multicast rate thresholds that drop or err-disable when a storm builds. |
| ARP / ND suppression | The leaf answers ARP/ND locally from the EVPN MAC/IP table instead of flooding across the WAN — kills most broadcast that would otherwise stretch. |
| EVPN loop protection | MAC mobility sequence numbers detect a MAC flapping between sites; MAC duplication detection freezes a looping/duplicated MAC. |
| Split-horizon / DF election | On multihomed segments (EVPN ESI), split-horizon filtering prevents a frame echoing back; the Designated Forwarder rule stops duplicate BUM delivery. |
| Keep STP out of the WAN | Do not extend spanning tree across the DCI. Use EVPN multihoming (ESI-LAG) for redundancy so a remote STP event never propagates. |
Independent of L2-vs-L3, you choose what carries the traffic between the border leaves. The overlay (EVPN) can generally ride any of these underlays.
| Transport | Characteristics | Use when |
|---|---|---|
| MPLS L3VPN | Provider or self-built VPNv4/v6; per-VRF isolation; mature TE via RSVP or SR. | Existing MPLS WAN; want VRF separation and SLAs. |
| Segment Routing / SR-TE | Source-routed paths, no per-flow state in the core; explicit low-latency or disjoint paths, fast reroute (TI-LFA). | Need latency-engineered or path-diverse DCI. See SR-TE theory. |
| Internet + IPsec / GRE | Cheap, ubiquitous, but variable latency/loss and no SLA; encryption mandatory. | Cost-sensitive DR/backup; SD-WAN overlays. |
| Dedicated DWDM / dark fiber | Lowest latency, highest capacity, full control; expensive and distance-limited. | Metro/campus DCI, sync replication, latency-critical stretch. |
The BGP address families that carry all of this (EVPN, VPNv4/v6, and the mechanics of RD/RT) are covered in BGP multiprotocol theory.
The reason modern DCI is tractable is that one control plane — MP-BGP EVPN — spans both the DC fabric and the WAN. The same address family that distributes MAC/IP inside the fabric also carries the DCI routes, so the border leaf just re-originates or stitches routes at the boundary.
| EVPN route type | Carries | Role in DCI |
|---|---|---|
| Type-2 | MAC + MAC/IP | Host reachability for L2 stretch and ARP/ND suppression; MAC mobility for VM moves. |
| Type-3 | Inclusive Multicast (IMET) | Builds the BUM flooding tree per stretched bridge domain. |
| Type-5 | IP Prefix | The L3 interconnect workhorse — advertise subnets and host routes across the WAN without extending L2. |
| Type-1 / Type-4 | Ethernet A-D / ESI | Multihoming: fast withdrawal, split-horizon, DF election on ESI-LAG. |
Terminating the overlay at the border leaf gives you independent failure domains, independent VNI/RT namespaces, per-site BUM containment, and a natural place to apply policy and re-write next-hops. A single flat end-to-end overlay is simpler to draw but couples the two DCs so tightly that a storm, a MAC loop, or a config error in one becomes an outage in both.
Running both DCs live means deciding which site a client reaches and which site a flow returns through. The tools:
The asymmetric-routing / stateful-device trap. With active-active and anycast, traffic can ingress DC1 but egress DC2 (or return via a different path). If a stateful device — firewall, NAT, or load balancer — sits in the path, it sees only half the flow and drops it, because the reverse packets hit a different box with no session state. Fixes: pin flows to a site (host routes + ingress optimization), use stateful firewall clustering / state sync across DCs, or design the stateful tier to be strictly per-site with symmetric hair-pinning. This is a favorite senior-level follow-up.