Layer 2 over a routed fabric
Stretch a layer-2 segment across a routed leaf-spine fabric: VXLAN for the data plane, BGP EVPN for the control plane. Built with FRR: the underlay, VNIs, type-2 and type-3 routes, route targets, and the VXLAN packets on the wire.
By the end you will be able to:
- Explain why data centres route the fabric but still need layer 2
- Describe VXLAN encapsulation, VNIs and VTEPs
- Configure an eBGP underlay and the EVPN address family on FRR
- Read type-2 and type-3 EVPN routes and route targets
- Follow a frame through the fabric in a packet capture
Prerequisites: BGP on loopbacks vs interface addresses, Switching basics: layer 2, VLANs and IP addressing
Data-centre networks have two wishes that pull against each other:
- Routed, everywhere. A fabric of leaf and spine switches running a routing protocol on every link: no spanning tree, every link in use, any switch can fail. (The switching basics lesson showed what a layer-2 loop does.)
- Layer 2, sometimes. Some applications (and virtual machines that move between hosts) need two machines in the same subnet, even when they're plugged into different leaves.
VXLAN reconciles them. Each layer-2 segment gets a 24-bit VNI (VXLAN Network Identifier), and the leaves (VTEPs, VXLAN tunnel endpoints) carry its Ethernet frames across the routed fabric inside UDP packets (port 4789). The fabric only ever routes IP between leaf loopbacks.
That leaves one question: how does each leaf know which MAC addresses live behind which other leaf? Early VXLAN flooded unknown traffic to every leaf and learnt from what came back, like a giant switch. EVPN answers it with BGP instead: each leaf advertises the MAC addresses it has learnt locally, as BGP routes. Remote MACs are installed from the control plane, never learnt by flooding.
This lesson's lab is a minimal fabric made of FRR containers: one spine (AS65000) and two leaves (AS65001, AS65002) with an eBGP underlay. VNI 100 is stretched across both leaves, with host1 (10.100.0.1) behind leaf1 and host2 (10.100.0.2) behind leaf2, the same subnet across a routed network. Every output comes from that lab.
- Underlay. eBGP between spine and leaves carries the leaf loopbacks: the VTEP addresses.
- EVPN routes. leaf1 advertises a type-3 route (VTEP for VNI 100) and a type-2 route for host1's MAC. The spine passes them on with the next hop unchanged.
- Frames inside UDP. host1's ARP and pings to host2 are wrapped in VXLAN (UDP 4789) from 10.0.0.11 to 10.0.0.12, crossing the spine as ordinary routed packets.
Check your understanding
What does EVPN replace in VXLAN networks?
- Flood-and-learn: remote MAC addresses are advertised in BGP instead of learnt from flooded traffic
- The UDP encapsulation
- The underlay routing
- Spanning tree inside each leaf