TTR Networks

Intermediate · Two Debian routers: VRRP and connection sync · section 9 of 9

Build the lab yourself

✓ Lab-tested 4 November 2026 on Debian 13.7, keepalived 2.3.3, conntrackd 1.4.8, nftables 1.1.3, systemd 257

The lab runs on any machine with Docker, including an Apple Silicon Mac. The two routers are Debian 13 with systemd, so keepalived, conntrackd, nftables and networkd behave as on real hardware; the ISP, the web server and the staff PC are plain Debian containers.

  1. Put the files from this page in one directory, with each router's files in r1/ and r2/, and stream.py (the long-lived TCP stream) next to lab-setup.sh.
  2. docker compose -f lab-compose.yaml up -d --build, then ./lab-setup.sh. It addresses the ISP, web server and staff PC, starts the stream server, and leaves forwarding off on the routers, as on a fresh install.
  3. For each router, copy its .network and .netdev files to /etc/systemd/network/, 99-router.conf to /etc/sysctl.d/ and nftables.conf to /etc/, then systemctl enable --now systemd-networkd nftables and sysctl --system.
  4. Copy keepalived.conf and conntrackd.conf to /root/ on each router, and follow the lesson.

To watch a failover, start the stream on the staff PC with timestamps: nc 198.51.100.10 9000 | while IFS= read -r l; do echo "$(date +%T.%3N) $l"; done, and kill a router from the host with docker compose -f lab-compose.yaml kill r1. Bring it back with docker compose -f lab-compose.yaml start r1.

The routers have fixed MAC addresses in lab-compose.yaml, so a restarted container is like a rebooted machine, not a new one.

lab-compose.yaml
# Two Debian routers: the lab.  docker compose -f lab-compose.yaml up -d --build && ./lab-setup.sh
name: ha-lab
x-router: &router
  build: { context: ., dockerfile: Dockerfile.router }
  image: ttr-ha-router:13
  privileged: true
  cgroup: host                          # let systemd run as PID 1
  volumes: ["/sys/fs/cgroup:/sys/fs/cgroup:rw"]
  tmpfs: ["/run", "/run/lock"]
x-host: &host
  build: { context: ., dockerfile: Dockerfile.host }
  image: ttr-ha-host:13
  privileged: true
services:
  r1: { <<: *router, hostname: r1, networks: { wan: { interface_name: eth0, priority: 100, mac_address: "02:00:00:00:01:00" }, trunk: { interface_name: eth1, priority: 90, mac_address: "02:00:00:00:01:01" }, sync: { interface_name: eth2, mac_address: "02:00:00:00:01:02" } } }
  r2: { <<: *router, hostname: r2, networks: { wan: { interface_name: eth0, priority: 100, mac_address: "02:00:00:00:02:00" }, trunk: { interface_name: eth1, priority: 90, mac_address: "02:00:00:00:02:01" }, sync: { interface_name: eth2, mac_address: "02:00:00:00:02:02" } } }
  isp:   { <<: *host, hostname: isp,   networks: { wan: { interface_name: eth0, priority: 100 }, internet: { interface_name: eth1 } } }
  web:   { <<: *host, hostname: web,   networks: { internet: { interface_name: eth0 } } }
  staff: { <<: *host, hostname: staff, networks: { trunk: { interface_name: eth0 } } }
# Layer-2-only networks. "trunk" is the office switch (staff tags VLAN 10 itself);
# "sync" is a cable between the two routers, for conntrackd.
networks:
  wan:      { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:d1::/64" }] } }
  trunk:    { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:d2::/64" }] } }
  sync:     { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:d3::/64" }] } }
  internet: { enable_ipv4: false, enable_ipv6: true, ipam: { config: [{ subnet: "fd00:d4::/64" }] } }
lab-setup.sh
#!/bin/sh
# Address everything except the routers (their configuration is the lesson). Run after `docker compose up`.
x() { docker compose -f lab-compose.yaml exec -T "$1" sh -c "$2"; }
for n in r1 r2 isp web staff; do x $n 'sysctl -qw net.ipv6.conf.all.disable_ipv6=1'; done
# Docker Desktop turns forwarding on in every container; a fresh Debian install has it off.
for n in r1 r2; do x $n 'sysctl -qw net.ipv4.ip_forward=0'; done
x isp   'ip addr add 203.0.113.1/24 dev eth0; ip addr add 198.51.100.1/24 dev eth1; sysctl -qw net.ipv4.ip_forward=1'
x web   'ip addr add 198.51.100.10/24 dev eth0; ip route add default via 198.51.100.1'
docker compose -f lab-compose.yaml cp stream.py web:/usr/local/bin/stream.py >/dev/null 2>&1
x web   'nohup python3 /usr/local/bin/stream.py > /dev/null 2>&1 &'
# A staff PC on VLAN 10, whose gateway is the routers' shared address, 10.10.10.1.
x staff 'ip link add link eth0 name eth0.10 type vlan id 10 && ip link set eth0.10 up && ip addr add 10.10.10.20/24 dev eth0.10 && ip route add default via 10.10.10.1'
stream.py
#!/usr/bin/env python3
# A long-lived TCP connection to watch during a failover: every 0.2 s, send a counter and the time.
import socketserver, time
class Stream(socketserver.BaseRequestHandler):
    def handle(self):
        n = 0
        while True:
            n += 1
            self.request.sendall(f"{n} {time.strftime('%H:%M:%S')}\n".encode())
            time.sleep(0.2)
socketserver.ThreadingTCPServer.allow_reuse_address = True
socketserver.ThreadingTCPServer(("", 9000), Stream).serve_forever()
Dockerfile.router
# The two routers: Debian 13 with systemd, udev and D-Bus (as in "Debian as an office router"),
# plus keepalived for VRRP and conntrackd to share connection state.
FROM debian:trixie-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
      systemd systemd-sysv udev dbus nftables keepalived conntrackd conntrack netbase iproute2 iputils-ping \
      tcpdump procps less \
 && rm -rf /var/lib/apt/lists/* \
 && systemctl mask systemd-modules-load.service getty@.service autovt@.service getty-static.service
CMD ["/sbin/init"]
  • Line 5: keepalived and conntrackd, and netbase for /etc/protocols, which conntrackd's filter needs.
Dockerfile.host
# Office PCs, servers and the internet: plain Debian 13 with network tools and a DHCP client.
FROM debian:trixie-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
      iproute2 iputils-ping tcpdump curl python3 netcat-openbsd procps dnsutils dhcpcd-base \
 && rm -rf /var/lib/apt/lists/*
CMD ["sleep", "infinity"]
r1/10-wan.network
# The internet side: this router's own address. The shared address, 203.0.113.4, is keepalived's.
[Match]
Name=eth0

[Network]
Address=203.0.113.2/24
Gateway=203.0.113.1
r1/20-trunk.network
# The office side: the trunk to the switch, carrying VLAN 10 (staff).
[Match]
Name=eth1

[Network]
VLAN=staff
LinkLocalAddressing=no
r1/30-staff.netdev
# VLAN 10: staff PCs. Their gateway is the shared address, 10.10.10.1.
[NetDev]
Name=staff
Kind=vlan

[VLAN]
Id=10
r1/30-staff.network
# This router's own address on VLAN 10; keepalived adds 10.10.10.1 on the active router.
[Match]
Name=staff

[Network]
Address=10.10.10.2/24
r1/40-sync.network
# A direct cable to the other router, for conntrackd.
[Match]
Name=eth2

[Network]
Address=10.99.0.1/30
LinkLocalAddressing=no
r2/10-wan.network
# The internet side: this router's own address. The shared address, 203.0.113.4, is keepalived's.
[Match]
Name=eth0

[Network]
Address=203.0.113.3/24
Gateway=203.0.113.1
r2/30-staff.network
# This router's own address on VLAN 10; keepalived adds 10.10.10.1 on the active router.
[Match]
Name=staff

[Network]
Address=10.10.10.3/24
r2/40-sync.network
# A direct cable to the other router, for conntrackd.
[Match]
Name=eth2

[Network]
Address=10.99.0.2/30
LinkLocalAddressing=no
r2/keepalived.conf
# /etc/keepalived/keepalived.conf on r2
global_defs {
    router_id r2
    enable_script_security
    script_user root
}

# Is the ISP reachable from here? Two failed pings in a row and this router steps down.
vrrp_script isp_reachable {
    script "/usr/bin/ping -c 1 -W 1 203.0.113.1"
    interval 2
    fall 2
    rise 2
}

# Both addresses move together: a router that holds one must hold the other.
vrrp_sync_group OFFICE {
    group {
        STAFF
        WAN
    }
}

vrrp_instance STAFF {
    state BACKUP
    interface staff
    virtual_router_id 10
    priority 100
    advert_int 1
    virtual_ipaddress {
        10.10.10.1/24
    }
    track_script {
        isp_reachable
    }
}

vrrp_instance WAN {
    state BACKUP
    interface eth0
    virtual_router_id 20
    priority 100
    advert_int 1
    virtual_ipaddress {
        203.0.113.4/24
    }
}
r2/conntrackd.conf
# /etc/conntrackd/conntrackd.conf on r2: send our connection table to the other router, and
# put theirs straight into our kernel, so either router can carry on any connection.
Sync {
	Mode FTFW {
		DisableExternalCache On
	}
	UDP {
		IPv4_address 10.99.0.2
		IPv4_Destination_Address 10.99.0.1
		Port 3780
		Interface eth2
		Checksum on
	}
}

General {
	HashSize 8192
	HashLimit 65535
	Syslog on
	LockFile /var/lock/conntrackd.lock
	UNIX {
		Path /var/run/conntrackd.sock
	}
	Systemd on
	Filter From Userspace {
		Protocol Accept {
			TCP
			UDP
			ICMP
		}
		Address Ignore {
			IPv4_address 127.0.0.1
			IPv4_address 10.99.0.0/30
		}
	}
}
/etc/sysctl.d/99-router.conf
# /etc/sysctl.d/99-router.conf: route between the WAN and the VLANs.
net.ipv4.ip_forward = 1

Where next

  • Debian as an office router: interfaces, VLANs and a firewallPrerequisite · Intermediate · 45 min

    Turn a fresh Debian 13 install into an office router: systemd-networkd for the WAN and three VLANs on one trunk, IP forwarding, an nftables policy between VLANs with NAT, protecting the router itself, and a configuration that survives a reboot.

  • Office gateway: DHCP, DNS and VRRP on VyOSAlso covers VRRP, high availability · Beginner · 40 min

    Two VyOS routers serving one office LAN: VRRP for a gateway that survives a router failure, DHCP with split pools and a printer reservation, and DNS forwarding. Then a real failover, the DNS outage it exposed, and the fix.

  • nftables and iptables on Debian: host and router firewallsAlso covers debian, nftables · Intermediate · 60 min

    Build a Debian firewall with nftables, from a host firewall to an office router with NAT and port forwards: safe changes with a timed rollback, sets and maps, logging and packet tracing, and what iptables really is on Debian 13.

  • OpenVPN on Debian: users, routing and revocationAlso covers debian, nftables · Intermediate · 60 min

    Install and run an OpenVPN server on Debian with your own certificate authority: one certificate per user, fixed addresses, routing to an office and a branch LAN, split and full tunnels, tls-crypt, revoking users, and the mistakes that lock everyone out.