TTR Networks

Intermediate · Setting up VyOS with basic BGP · section 9 of 9

Build the lab yourself

✓ Lab-tested 17 October 2026 on VyOS 2026.10.16-0024-rolling, FRR 10.2.1

The lab runs VyOS as a QEMU virtual machine inside a container, exactly as in the Packets That Never Come Back lesson. Get the ISO with that lesson's get-vyos.sh, copy FRR's daemons file out of the FRR image (enable bgpd), save the files below in one directory, then docker compose up -d --build. VyOS takes a few minutes to boot under emulation. Log in on the console as vyos/vyos and paste the stage files in configuration mode, committing after each one. For stage 5, generate your own key on the host first (docker compose exec host ssh-keygen -t ed25519) and use its public key instead of the one shown.

lab-compose.yaml
# Needs ./vyos (from the tcpdump-and-vyos lesson's get-vyos.sh) and ./daemons (copy /etc/frr/daemons out of the FRR image, set bgpd=yes). Then: docker compose up -d --build
# VyOS console: docker compose exec -it vyos socat -,raw,echo=0,escape=0x1d UNIX-CONNECT:/run/console.sock
name: vyos-bgp-lab
x-frr: &frr
  image: quay.io/frrouting/frr:10.2.1
  privileged: true
services:
  vyos:
    build:
      context: .
      dockerfile_inline: |
        FROM debian:bookworm-slim
        RUN apt-get update && apt-get install -y --no-install-recommends qemu-system-x86 iproute2 socat
        COPY start-vyos.sh /usr/local/bin/start-vyos.sh
        CMD ["/usr/local/bin/start-vyos.sh"]
    cap_add: [NET_ADMIN]
    devices: ["/dev/net/tun:/dev/net/tun"]
    volumes: ["./vyos:/vyos:ro"]
    networks: { transit: { interface_name: eth0 }, ix: { interface_name: eth1 }, lan: { interface_name: eth2 } }
  upstream:
    <<: *frr
    volumes: ["./daemons:/etc/frr/daemons:ro", "./upstream.conf:/etc/frr/frr.conf"]
    networks: { transit: { interface_name: eth0 } }
  rs1:
    <<: *frr
    volumes: ["./daemons:/etc/frr/daemons:ro", "./rs.conf:/etc/frr/frr.conf"]
    networks: { ix: { interface_name: eth0 } }
  member:
    <<: *frr
    volumes: ["./daemons:/etc/frr/daemons:ro", "./member.conf:/etc/frr/frr.conf"]
    networks: { ix: { interface_name: eth0 } }
  host:
    build:
      dockerfile_inline: |
        FROM debian:bookworm-slim
        RUN apt-get update && apt-get install -y --no-install-recommends iproute2 iputils-ping traceroute netcat-openbsd openssh-client
        CMD ["sh", "-c", "ip addr add 198.51.100.10/25 dev eth0 && ip route replace default via 198.51.100.1 && exec sleep infinity"]
    privileged: true
    networks: { lan: { interface_name: eth0 } }
networks:
  transit: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:e::/64" }] } }
  ix: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:f::/64" }] } }
  lan: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:10::/64" }] } }
  • Lines 8–19: The VyOS VM container with three NICs: transit, IX and LAN, in that order.
  • Lines 20–31: FRR containers play the upstream, the route server and another IX member.
  • Lines 32–39: A Linux host on R1's LAN, using R1 as its default gateway.
  • Lines 40–43: Layer-2-only networks: every IPv4 address comes from the configs.
start-vyos.sh
#!/bin/sh
# Bridge every container interface (eth0, eth1, …) to a tap, then boot VyOS with one virtio NIC per tap.
# VM NIC n gets MAC 52:54:00:00:0n:01, so VyOS's ethN matches the container's ethN.
set -e
NETDEVS=""
n=0
for ifc in $(ls /sys/class/net | grep -E '^eth[0-9]+$' | sort -V); do
  ip link add "br$n" type bridge
  ip link set "$ifc" master "br$n"
  ip tuntap add "tap$n" mode tap
  ip link set "tap$n" master "br$n"
  ip link set "tap$n" up
  ip link set "br$n" up
  # Pin the VM's MAC to its tap and stop learning on the Docker side: Docker reflects the VM's
  # broadcasts back, which would otherwise teach the bridge the VM's MAC on the wrong port.
  bridge link set dev "$ifc" learning off
  bridge fdb replace "52:54:00:00:0$n:01" dev "tap$n" master static
  NETDEVS="$NETDEVS -netdev tap,id=n$n,ifname=tap$n,script=no,downscript=no -device virtio-net-pci,netdev=n$n,mac=52:54:00:00:0$n:01"
  n=$((n + 1))
done
exec qemu-system-x86_64 -m 2048 -smp 2 -cpu max -accel tcg,thread=multi \
  -kernel /vyos/vmlinuz -initrd /vyos/initrd.img \
  -append "boot=live components hostname=vyos username=live nopersistence noautologin nonetworking union=overlay console=ttyS0,115200 net.ifnames=0 biosdevname=0" \
  -cdrom /vyos/vyos.iso $NETDEVS \
  -display none -serial unix:/run/console.sock,server=on,wait=off -monitor none
  • Lines 7–13: One bridge and tap per container interface, so each VM NIC lands on the matching Docker network.
  • Lines 14–17: Pin the VM's MAC to its tap and turn off learning on the Docker side, or replies to the VM are intermittently lost.
  • Line 21: Software emulation. On an x86 Linux host, -enable-kvm makes it much faster.
upstream.conf (FRR)
frr defaults traditional
hostname upstream
!
interface eth0
 ip address 203.0.113.0/31
!
interface lo
 ip address 203.0.113.130/32
!
ip route 203.0.113.128/25 Null0
!
ip prefix-list CUSTOMER-PREFIXES seq 10 permit 198.51.100.0/24
!
route-map CUSTOMER-IN permit 10
 match ip address prefix-list CUSTOMER-PREFIXES
exit
!
route-map CUSTOMER-OUT permit 10
exit
!
router bgp 64496
 bgp router-id 203.0.113.0
 neighbor 203.0.113.1 remote-as 64500
 neighbor 203.0.113.1 description CUSTOMER-R1
 !
 address-family ipv4 unicast
  network 203.0.113.128/25
  neighbor 203.0.113.1 default-originate
  neighbor 203.0.113.1 route-map CUSTOMER-IN in
  neighbor 203.0.113.1 route-map CUSTOMER-OUT out
 exit-address-family
exit
!
  • Lines 7–10: A loopback in the upstream's /25: the target for the traceroute.
  • Lines 12–16: The upstream's customer filter: it only accepts our /24. Real providers build this from your IRR objects.
  • Line 28: Sends us a default route.
rs.conf (FRR)
frr defaults traditional
hostname rs1
!
interface eth0
 ip address 192.0.2.1/24
!
router bgp 64511
 bgp router-id 192.0.2.1
 no bgp ebgp-requires-policy
 neighbor 192.0.2.25 remote-as 64500
 neighbor 192.0.2.25 description R1-AS64500
 neighbor 192.0.2.40 remote-as 64502
 neighbor 192.0.2.40 description MEMBER-AS64502
 !
 address-family ipv4 unicast
  neighbor 192.0.2.25 route-server-client
  neighbor 192.0.2.40 route-server-client
 exit-address-family
exit
!
  • Lines 16–17: route-server-client: don't add AS64511 to the path and don't change the next hop.
member.conf (FRR)
frr defaults traditional
hostname member
!
interface eth0
 ip address 192.0.2.40/24
!
interface lo
 ip address 203.0.113.65/32
!
ip route 203.0.113.64/26 Null0
ip route 10.99.0.0/16 Null0
!
router bgp 64502
 bgp router-id 192.0.2.40
 no bgp ebgp-requires-policy
 no bgp enforce-first-as
 neighbor 192.0.2.1 remote-as 64511
 neighbor 192.0.2.1 description IX-RS1
 !
 address-family ipv4 unicast
  network 203.0.113.64/26
  network 10.99.0.0/16
 exit-address-family
exit
!
  • Lines 11–22: The deliberate mistake: a private range, announced along with the member's real prefix.
  • Line 16: Needed because routes from the route server start with the originating member's AS, not 64511.

Where next

  • BGP communities, prepending and RTBHBuilds on this lesson · Intermediate · 45 min

    Tag routes with large communities, steer inbound traffic with AS-path prepending (and see why providers' local preference limits it), and stop a DDoS at your provider's edge with remotely triggered blackholing. Built on VyOS with two transit providers, including the two RTBH mistakes the lab made first.

  • Edge and core: building out a VyOS networkBuilds on this lesson · Intermediate · 60 min

    Split one BGP router into edge routers that face the outside world and a core that carries traffic inside: loopbacks, OSPF for infrastructure, iBGP between loopbacks, next-hop-self, and what happens when a link fails. Built and broken on three real VyOS routers.

  • BGP FundamentalsPrerequisite · Intermediate · 75 min

    How the Border Gateway Protocol glues the internet together: autonomous systems, sessions, path attributes, best-path selection, iBGP scaling and a hands-on eBGP lab with real FRR configuration.

  • BGP on loopbacks vs interface addressesAlso covers BGP · Intermediate · 45 min

    Why BGP sessions between loopbacks survive link failures and use parallel links, and what they need to work: an IGP or static routes, update-source, the eBGP connected check, and BFD. Every failure is reproduced on two real VyOS routers.