A VNI and its routes
On each leaf, VNI 100 is built from three Linux pieces:
- A bridge, br100: the layer-2 segment on this leaf.
- A VXLAN interface, vxlan100: VNI 100, UDP 4789, sourced from the leaf's loopback, with
nolearning, because EVPN will tell it where remote MACs live. It's a port on the bridge. - The host, plugged into the bridge (here a network namespace on a veth "cable").
FRR notices the VNI in the kernel and immediately advertises two kinds of EVPN route:
- Type 3, inclusive multicast: "I'm a VTEP for VNI 100, at 10.0.0.11." Every leaf with the VNI uses these to build its flood list, the VTEPs that should receive broadcast and unknown traffic. Without multicast in the underlay, the leaf copies each such frame to every VTEP on the list: head-end replication (HER).
- Type 2, MAC/IP: "MAC 02:00:00:00:01:01 is behind 10.0.0.11." One per host MAC learnt on a local port.
leaf1's EVPN table has both of its own routes and both of leaf2's, with next hop 10.0.0.12 (unchanged by the spine) and AS path 65000 65002. show evpn vni reports two MACs (one local, one remote) and one remote VTEP.
Route targets
Each route carries a route target (RT) that says which VNI it belongs to. FRR derives them automatically as AS:VNI, so leaf1's are 65001:100 and leaf2's 65002:100, and leaf1's import RT is only 65001:100. Yet leaf1 imported leaf2's routes anyway. For auto-derived route targets, FRR matched only the VNI part on import, which is what lets eBGP fabrics with one AS per leaf work without extra configuration. Not every implementation does this. In a mixed-vendor fabric, configure the same route target (say 65000:100) explicitly on every leaf.
#!/bin/sh
# On a leaf: VNI 100 as a bridge with a VXLAN port, and one host (a network namespace) on the bridge.
# Usage inside the leaf: vxlan-setup.sh <vtep-loopback> <host-name> <host-ip> <host-mac>
set -e
ip link add br100 type bridge
ip link set br100 up
ip link add vxlan100 type vxlan id 100 dstport 4789 local "$1" nolearning
ip link set vxlan100 master br100
ip link set vxlan100 up
ip netns add "$2"
ip link add "$2-port" type veth peer name eth0 netns "$2"
ip link set "$2-port" master br100 up
ip -n "$2" link set eth0 address "$4"
ip -n "$2" addr add "$3" dev eth0
ip -n "$2" link set eth0 up
ip -n "$2" link set lo up- Lines 5–6: The segment's bridge.
- Line 7: VXLAN for VNI 100, from the loopback, no flood-and-learn.
- Lines 8–9: A port on the bridge.
- Lines 10–16: A host on the segment.
leaf1# show evpn vni
VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs Tenant VRF
100 L2 vxlan100 2 0 1 default
leaf1# show bgp l2vpn evpn
BGP table version is 2, local router ID is 10.0.0.11
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 10.0.0.11:2
*> [2]:[0]:[48]:[02:00:00:00:01:01]
10.0.0.11 32768 i
ET:8 RT:65001:100
*> [3]:[0]:[32]:[10.0.0.11]
10.0.0.11 32768 i
ET:8 RT:65001:100
Route Distinguisher: 10.0.0.12:2
*> [2]:[0]:[48]:[02:00:00:00:02:02]
10.0.0.12 0 65000 65002 i
RT:65002:100 ET:8
*> [3]:[0]:[32]:[10.0.0.12]
10.0.0.12 0 65000 65002 i
RT:65002:100 ET:8
Displayed 4 out of 4 total prefixes- Line 3: VNI 100: two MACs, one remote VTEP.
- Lines 16–18: Type 2: host1's MAC, behind leaf1.
- Lines 19–21: Type 3: leaf1 is a VTEP for VNI 100.
- Lines 23–28: leaf2's routes: next hop leaf2's loopback, via the spine.
leaf1# show evpn mac vni 100
Number of MACs (local and remote) known for this VNI: 2
Flags: N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC Type Flags Intf/Remote ES/VTEP VLAN Seq #'s
02:00:00:00:01:01 local host1-port 0/0
02:00:00:00:02:02 remote 10.0.0.12 0/0
leaf1# show evpn vni 100
VNI: 100
Type: L2
Vlan: 1
Bridge: br100
Tenant VRF: default
VxLAN interface: vxlan100
VxLAN ifIndex: 13
SVI interface: br100
SVI ifIndex: 12
Local VTEP IP: 10.0.0.11
Mcast group: 0.0.0.0
Remote VTEPs for this VNI:
10.0.0.12 flood: HER
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
Advertise-svi-macip: No- Line 5: Local: on host1's port.
- Line 6: Remote: behind VTEP 10.0.0.12.
- Line 20: Flood list: head-end replication to leaf2.
leaf1# show bgp l2vpn evpn vni 100
VNI: 100 (known to the kernel)
Type: L2
Tenant-Vrf: default
RD: 10.0.0.11:2
Originator IP: 10.0.0.11
Mcast group: 0.0.0.0
MAC-VRF Site-of-Origin:
Advertise-gw-macip : Disabled
Advertise-svi-macip : Disabled
SVI interface : br100
Import Route Target:
65001:100
Export Route Target:
65001:100- Lines 12–13: leaf1 imports 65001:100 only, yet leaf2's 65002:100 routes were imported: FRR matched the auto-derived RT on the VNI.
Check your understanding
Why does the spine use attribute-unchanged next-hop for EVPN?
- The next hop is the VTEP; rewriting it to the spine would send VXLAN traffic to a switch with no VTEP
- To make the spine a VTEP
- eBGP requires it
- To hide the leaves' addresses