TTR Networks

Intermediate · BGP on loopbacks vs interface addresses · section 8 of 8

Build the lab yourself

✓ Lab-tested 17 October 2026 on VyOS 2026.10.16-0024-rolling

Two VyOS VMs, each with two NICs. Put ./vyos (from the Packets That Never Come Back lesson's get-vyos.sh) next to these files and run docker compose up -d --build. Log in on each console as vyos/vyos and apply the configuration blocks in order. Break link A with set interfaces ethernet eth0 disable on R2, and restore it with delete interfaces ethernet eth0 disable.

lab-compose.yaml
# Needs ./vyos (from the tcpdump-and-vyos lesson's get-vyos.sh). Then: docker compose up -d --build
# Console: docker compose exec -it r1 socat -,raw,echo=0,escape=0x1d UNIX-CONNECT:/run/console.sock
name: vyos-loopbacks-lab
x-vyos: &vyos
  build:
    context: .
    dockerfile_inline: |
      FROM debian:bookworm-slim
      RUN apt-get update && apt-get install -y --no-install-recommends qemu-system-x86 iproute2 socat
      COPY start-vyos.sh /usr/local/bin/start-vyos.sh
      CMD ["/usr/local/bin/start-vyos.sh"]
  cap_add: [NET_ADMIN]
  devices: ["/dev/net/tun:/dev/net/tun"]
  volumes: ["./vyos:/vyos:ro"]
services:
  r1:
    <<: *vyos
    environment: { VYOS_MEM: "1536", VYOS_SMP: "1", VM_ID: "01" }
    networks: { link-a: { interface_name: eth0 }, link-b: { interface_name: eth1 } }
  r2:
    <<: *vyos
    environment: { VYOS_MEM: "1536", VYOS_SMP: "1", VM_ID: "02" }
    networks: { link-a: { interface_name: eth0 }, link-b: { interface_name: eth1 } }
networks:
  link-a: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:30::/64" }] } }
  link-b: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:31::/64" }] } }
  • Lines 16–23: Two VMs with unique VM_IDs, so their MAC addresses differ. One vCPU each: multi-vCPU guests occasionally locked up under emulation on an ARM Mac.
  • Lines 24–26: Link A and link B: layer-2-only networks.
start-vyos.sh
#!/bin/sh
# Bridge every container interface (eth0, eth1, …) to a tap, then boot VyOS with one virtio NIC per tap.
# VM NIC n gets MAC 52:54:00:<VM_ID>:0n:01, so VyOS's ethN matches the container's ethN. Give each VM its own VM_ID.
set -e
NETDEVS=""
n=0
for ifc in $(ls /sys/class/net | grep -E '^eth[0-9]+$' | sort -V); do
  ip link add "br$n" type bridge
  ip link set "$ifc" master "br$n"
  ip tuntap add "tap$n" mode tap
  ip link set "tap$n" master "br$n"
  ip link set "tap$n" up
  ip link set "br$n" up
  # Pin the VM's MAC to its tap and stop learning on the Docker side: Docker reflects the VM's
  # broadcasts back, which would otherwise teach the bridge the VM's MAC on the wrong port.
  bridge link set dev "$ifc" learning off
  bridge fdb replace "52:54:00:${VM_ID:-00}:0$n:01" dev "tap$n" master static
  NETDEVS="$NETDEVS -netdev tap,id=n$n,ifname=tap$n,script=no,downscript=no -device virtio-net-pci,netdev=n$n,mac=52:54:00:${VM_ID:-00}:0$n:01"
  n=$((n + 1))
done
exec qemu-system-x86_64 -m "${VYOS_MEM:-2048}" -smp "${VYOS_SMP:-2}" -cpu max -accel tcg,thread=multi \
  -kernel /vyos/vmlinuz -initrd /vyos/initrd.img \
  -append "boot=live components hostname=vyos username=live nopersistence noautologin nonetworking union=overlay console=ttyS0,115200 net.ifnames=0 biosdevname=0" \
  -cdrom /vyos/vyos.iso $NETDEVS \
  -display none -serial unix:/run/console.sock,server=on,wait=off -monitor none
  • Lines 14–17: Pin the VM's MAC to its tap and turn off learning on the Docker side.
  • Line 21: Memory and vCPU count from the compose file.

Where next

  • BGP FundamentalsPrerequisite · Intermediate · 75 min

    How the Border Gateway Protocol glues the internet together: autonomous systems, sessions, path attributes, best-path selection, iBGP scaling and a hands-on eBGP lab with real FRR configuration.

  • OSPF FundamentalsPrerequisite · Beginner · 70 min

    The link-state IGP that runs inside most networks: neighbour adjacencies, LSA flooding, SPF, costs, DR/BDR, areas and LSA types, a hands-on FRR lab, and how OSPF underpins BGP.

  • Edge and core: building out a VyOS networkAlso covers BGP, iBGP, OSPF · Intermediate · 60 min

    Split one BGP router into edge routers that face the outside world and a core that carries traffic inside: loopbacks, OSPF for infrastructure, iBGP between loopbacks, next-hop-self, and what happens when a link fails. Built and broken on three real VyOS routers.

  • EVPN and VXLAN: layer 2 over a routed fabricBuilds on this lesson · Advanced · 45 min

    Stretch a layer-2 segment across a routed leaf-spine fabric: VXLAN for the data plane, BGP EVPN for the control plane. Built with FRR: the underlay, VNIs, type-2 and type-3 routes, route targets, and the VXLAN packets on the wire.