Build the lab yourself
One VyOS VM and four FRR/Linux containers. Put ./vyos (from the Packets That Never Come Back lesson's get-vyos.sh) and FRR's daemons file (bgpd=yes) next to these files, run docker compose up -d --build, address the host as the compose file's comment shows, then apply R1's stages in order. upstream2's config is the same as upstream1's with its own addresses and AS64497.
# Needs ./vyos (from the tcpdump-and-vyos lesson's get-vyos.sh) and ./daemons (copy /etc/frr/daemons out of the FRR image, set bgpd=yes).
# Then: docker compose up -d --build, and on the host: ip addr add 198.51.100.10/25 dev eth0; ip route replace default via 198.51.100.1
name: vyos-communities-lab
x-frr: &frr
image: quay.io/frrouting/frr:10.2.1
privileged: true
services:
r1:
build:
context: .
dockerfile_inline: |
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends qemu-system-x86 iproute2 socat
COPY start-vyos.sh /usr/local/bin/start-vyos.sh
CMD ["/usr/local/bin/start-vyos.sh"]
cap_add: [NET_ADMIN]
devices: ["/dev/net/tun:/dev/net/tun"]
environment: { VYOS_MEM: "1536", VYOS_SMP: "1", VM_ID: "01" }
volumes: ["./vyos:/vyos:ro"]
networks: { transit-a: { interface_name: eth0 }, transit-b: { interface_name: eth1 }, lan: { interface_name: eth2 } }
upstream1:
<<: *frr
volumes: ["./daemons:/etc/frr/daemons:ro", "./upstream1.conf:/etc/frr/frr.conf"]
networks: { transit-a: { interface_name: eth0 }, up1-inet: { interface_name: eth1 } }
upstream2:
<<: *frr
volumes: ["./daemons:/etc/frr/daemons:ro", "./upstream2.conf:/etc/frr/frr.conf"]
networks: { transit-b: { interface_name: eth0 }, up2-inet: { interface_name: eth1 } }
internet:
<<: *frr
volumes: ["./daemons:/etc/frr/daemons:ro", "./internet.conf:/etc/frr/frr.conf"]
networks: { up1-inet: { interface_name: eth0 }, up2-inet: { interface_name: eth1 } }
host:
build:
dockerfile_inline: |
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends iproute2 iputils-ping traceroute
CMD ["sleep", "infinity"]
privileged: true
networks: { lan: { interface_name: eth0 } }
networks:
transit-a: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:50::/64" }] } }
transit-b: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:51::/64" }] } }
lan: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:52::/64" }] } }
up1-inet: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:53::/64" }] } }
up2-inet: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:54::/64" }] } }- Lines 8–20: R1: transit A, transit B and the LAN.
- Lines 21–32: Two transits and the internet AS behind them.
frr defaults traditional
hostname internet
!
interface eth0
ip address 203.0.113.5/31
!
interface eth1
ip address 203.0.113.7/31
!
interface lo
ip address 203.0.113.130/32
!
ip route 203.0.113.128/25 Null0
!
router bgp 64511
bgp router-id 203.0.113.130
no bgp ebgp-requires-policy
neighbor 203.0.113.4 remote-as 64496
neighbor 203.0.113.4 description UPSTREAM1
neighbor 203.0.113.6 remote-as 64497
neighbor 203.0.113.6 description UPSTREAM2
!
address-family ipv4 unicast
network 203.0.113.128/25
exit-address-family
exit
!- Lines 10–13: The internet's own prefix and a loopback in it, for traceroutes.
- Lines 18–21: Connected to both transits.
frr defaults traditional
hostname upstream2
!
interface eth0
ip address 203.0.113.2/31
!
interface eth1
ip address 203.0.113.6/31
!
! RTBH: anything tagged BLACKHOLE is sent to this discard next hop
ip route 192.0.2.1/32 Null0
!
ip prefix-list CUSTOMER-PREFIXES seq 10 permit 198.51.100.0/24
ip prefix-list CUSTOMER-BLACKHOLES seq 10 permit 198.51.100.0/24 ge 32
!
bgp community-list standard BLACKHOLE seq 10 permit blackhole
!
route-map CUSTOMER-IN permit 10
match ip address prefix-list CUSTOMER-BLACKHOLES
match community BLACKHOLE
set ip next-hop 192.0.2.1
set community no-export additive
exit
!
route-map CUSTOMER-IN permit 20
match ip address prefix-list CUSTOMER-PREFIXES
set local-preference 200
exit
!
router bgp 64497
bgp router-id 203.0.113.2
no bgp ebgp-requires-policy
neighbor 203.0.113.3 remote-as 64500
neighbor 203.0.113.3 description CUSTOMER-R1
neighbor 203.0.113.3 disable-connected-check
neighbor 203.0.113.7 remote-as 64511
neighbor 203.0.113.7 description PEER-INTERNET
!
address-family ipv4 unicast
neighbor 203.0.113.3 route-map CUSTOMER-IN in
neighbor 203.0.113.3 default-originate
exit-address-family
exit
!- Line 30: The same policy as upstream1, as AS64497.
Where next
- Setting up VyOS with basic BGPPrerequisite · Intermediate · 50 min
Build a VyOS edge router from scratch: one upstream transit provider, one Internet Exchange route server, and the import and export policy that keeps both safe. Every command and output comes from a real VyOS lab.
- Edge and core: building out a VyOS networkAlso covers BGP · Intermediate · 60 min
Split one BGP router into edge routers that face the outside world and a core that carries traffic inside: loopbacks, OSPF for infrastructure, iBGP between loopbacks, next-hop-self, and what happens when a link fails. Built and broken on three real VyOS routers.
- BGP on loopbacks vs interface addressesAlso covers BGP · Intermediate · 45 min
Why BGP sessions between loopbacks survive link failures and use parallel links, and what they need to work: an IGP or static routes, update-source, the eBGP connected check, and BFD. Every failure is reproduced on two real VyOS routers.
- Route reflectors on VyOSAlso covers BGP · Intermediate · 30 min
Replace an iBGP full mesh with route reflectors: see iBGP refuse to pass routes on, turn on reflection, read ORIGINATOR_ID and CLUSTER_LIST, confirm the reflector stays out of the forwarding path, and survive taking one reflector down.