Trunks and routing between VLANs
VLAN 10 and VLAN 20 now need a router between them. You could give R1 one cable per VLAN, but that doesn't scale. Instead, one cable carries all the VLANs: a trunk.
On a trunk, every frame carries an 802.1Q tag: four extra bytes after the source MAC address, holding the VLAN number. The switch adds the tag when it sends a frame onto the trunk and removes it when it delivers the frame to an access port. Devices on access ports never see tags; only switches and routers on trunks do.
R1 (router on a stick): one physical interface, eth0, with a subinterface per VLAN: eth0.10 (vif 10) gets 10.10.0.1/24 and eth0.20 gets 10.20.0.1/24. Each one is the default gateway for its VLAN. Give the PCs their default routes.
sw1: make eth4 a trunk: no untagged VLAN, and tagged VLANs 10 and... only 10. That's a deliberate mistake, and a very common one. pc1 reaches its gateway, but pc3 can't reach 10.20.0.1, and pc1 can't reach pc3. R1 itself answers with Destination Host Unreachable: its ARP requests for pc3 go out on VLAN 20, and the switch drops them on the trunk because VLAN 20 isn't allowed there.
Add VLAN 20 to the trunk, and everything works. The traceroute from pc1 shows one hop through the router, and a capture on the trunk shows the whole story of a single ping in four frames:
- pc1 → R1, tagged vlan 10: the echo request arrives from VLAN 10.
- R1 → pc3, tagged vlan 20: the same packet, routed, sent back out on VLAN 20.
- pc3 → R1, tagged vlan 20: the reply.
- R1 → pc1, tagged vlan 10.
The same packet crosses the same cable twice, in different VLANs. R1's ARP table shows each PC on its own subinterface, and the switch's MAC table now records which VLAN each MAC address was learnt in. (The entries without a VLAN are left over from the flat network, before VLAN filtering was turned on, and will age out.)
Routing between VLANs could equally happen on a layer 3 switch with one Switched Virtual Interface (SVI) per VLAN, instead of a separate router. The concepts are identical; only the box changes.
set system host-name 'R1'
set interfaces ethernet eth0 description 'TRUNK-TO-SW1'
set interfaces ethernet eth0 vif 10 address '10.10.0.1/24'
set interfaces ethernet eth0 vif 10 description 'VLAN10-STAFF'
set interfaces ethernet eth0 vif 20 address '10.20.0.1/24'
set interfaces ethernet eth0 vif 20 description 'VLAN20-PRINTERS'- Line 2: The physical port: no address of its own.
- Lines 3–4: eth0.10: VLAN 10's gateway.
- Lines 5–6: eth0.20: VLAN 20's gateway.
vyos@R1:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address MAC VRF MTU S/L Description
----------- ------------ ----------------- ------- ----- ----- ---------------
eth0 - 52:54:00:01:00:01 default 1500 u/u TRUNK-TO-SW1
eth0.10 10.10.0.1/24 52:54:00:01:00:01 default 1500 u/u VLAN10-STAFF
eth0.20 10.20.0.1/24 52:54:00:01:00:01 default 1500 u/u VLAN20-PRINTERS
lo 127.0.0.1/8 00:00:00:00:00:00 default 65536 u/u
::1/128- Line 5: The trunk port.
- Lines 6–7: One subinterface per VLAN, all with the same MAC address.
ip netns exec pc1 ip route add default via 10.10.0.1
ip netns exec pc2 ip route add default via 10.10.0.1
ip netns exec pc3 ip route add default via 10.20.0.1- Lines 1–3: Each PC points at R1's address in its own VLAN.
# Make eth4 (to R1) a trunk: tagged frames only, no untagged VLAN. The mistake: VLAN 20 forgotten.
bridge vlan del dev eth4 vid 1
bridge vlan add dev eth4 vid 10- Line 2: No untagged VLAN on the trunk.
- Line 3: VLAN 10 tagged… and VLAN 20 forgotten.
root@sw1:~# bridge vlan show dev eth4
port vlan-id
eth4 10
root@pc1:~# ping -c 2 10.10.0.1
PING 10.10.0.1 (10.10.0.1) 56(84) bytes of data.
64 bytes from 10.10.0.1: icmp_seq=1 ttl=64 time=8.99 ms
64 bytes from 10.10.0.1: icmp_seq=2 ttl=64 time=2.30 ms
--- 10.10.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1006ms
rtt min/avg/max/mdev = 2.300/5.644/8.989/3.344 ms
root@pc3:~# ping -c 2 10.20.0.1
PING 10.20.0.1 (10.20.0.1) 56(84) bytes of data.
From 10.20.0.13 icmp_seq=1 Destination Host Unreachable
From 10.20.0.13 icmp_seq=2 Destination Host Unreachable
--- 10.20.0.1 ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1020ms
pipe 2
root@pc1:~# ping -c 2 10.20.0.13
PING 10.20.0.13 (10.20.0.13) 56(84) bytes of data.
From 10.10.0.1 icmp_seq=1 Destination Host Unreachable
From 10.10.0.1 icmp_seq=2 Destination Host Unreachable
--- 10.20.0.13 ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1000ms
pipe 2- Line 3: The trunk only carries VLAN 10.
- Lines 4–11: pc1 reaches its gateway.
- Lines 14–15: pc3 can't reach its gateway: VLAN 20 never gets to R1.
- Lines 22–23: R1 can't find pc3 either: its ARP requests on VLAN 20 are dropped at the trunk.
# The forgotten VLAN
bridge vlan add dev eth4 vid 20- Line 2: Allow VLAN 20 on the trunk.
root@sw1:~# bridge vlan show
port vlan-id
eth1 10 PVID Egress Untagged
eth2 10 PVID Egress Untagged
eth3 20 PVID Egress Untagged
eth4 10
20
br0 1 PVID Egress Untagged
root@pc1:~# ping -c 1 10.20.0.13
PING 10.20.0.13 (10.20.0.13) 56(84) bytes of data.
64 bytes from 10.20.0.13: icmp_seq=1 ttl=63 time=1.29 ms
--- 10.20.0.13 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.286/1.286/1.286/0.000 ms
root@pc1:~# traceroute -n 10.20.0.13
traceroute to 10.20.0.13 (10.20.0.13), 30 hops max, 60 byte packets
1 10.10.0.1 1.070 ms 1.128 ms 1.128 ms
2 10.20.0.13 3.464 ms 3.465 ms 3.584 ms- Lines 6–7: The trunk carries both VLANs, tagged.
- Line 11: pc1 reaches pc3…
- Lines 18–19: …via R1: one routed hop.
root@sw1:~# timeout 8 tcpdump -lnei eth4 "vlan and icmp"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth4, link-type EN10MB (Ethernet), snapshot length 262144 bytes
05:27:35.588466 02:00:00:00:00:01 > 52:54:00:01:00:01, ethertype 802.1Q (0x8100), length 102: vlan 10, p 0, ethertype IPv4 (0x0800), 10.10.0.11 > 10.20.0.13: ICMP echo request, id 343, seq 1, length 64
05:27:35.588998 52:54:00:01:00:01 > 02:00:00:00:00:03, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4 (0x0800), 10.10.0.11 > 10.20.0.13: ICMP echo request, id 343, seq 1, length 64
05:27:35.589008 02:00:00:00:00:03 > 52:54:00:01:00:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4 (0x0800), 10.20.0.13 > 10.10.0.11: ICMP echo reply, id 343, seq 1, length 64
05:27:35.589175 52:54:00:01:00:01 > 02:00:00:00:00:01, ethertype 802.1Q (0x8100), length 102: vlan 10, p 0, ethertype IPv4 (0x0800), 10.20.0.13 > 10.10.0.11: ICMP echo reply, id 343, seq 1, length 64
[... 15 lines trimmed: the same traceroute probes and replies, tagged the same way ...]
19 packets captured
19 packets received by filter
0 packets dropped by kernel- Line 4: pc1 → R1, tagged VLAN 10 (ethertype 802.1Q).
- Line 5: R1 → pc3: the same packet, now tagged VLAN 20.
- Line 6: pc3's reply to R1, in VLAN 20.
- Line 7: R1 → pc1, back in VLAN 10.
vyos@R1:~$ show arp
Address Interface Link layer address State
---------- ----------- -------------------- ---------
10.20.0.13 eth0.20 02:00:00:00:00:03 REACHABLE
10.10.0.11 eth0.10 02:00:00:00:00:01 REACHABLE- Lines 4–5: Each PC on its own subinterface.
root@sw1:~# bridge fdb show br br0 | grep -v permanent
02:00:00:00:00:01 dev eth1 vlan 10 master br0
02:00:00:00:00:01 dev eth1 master br0
02:00:00:00:00:02 dev eth2 vlan 10 master br0
02:00:00:00:00:02 dev eth2 master br0
02:00:00:00:00:03 dev eth3 vlan 20 master br0
52:54:00:01:00:01 dev eth4 vlan 20 master br0
52:54:00:01:00:01 dev eth4 vlan 10 master br0
52:54:00:01:00:01 dev eth4 master br0- Line 2: Learnt in VLAN 10.
- Line 3: Left over from before VLAN filtering; it will age out.
- Lines 7–8: R1's single MAC address, learnt separately in each VLAN on the trunk.
Check your understanding
On the trunk capture, why does the same ICMP echo request appear twice?
- It goes up the trunk to R1 in VLAN 10, and R1 routes it back down the same trunk in VLAN 20
- It's a retransmission
- The switch flooded it
- Spanning tree duplicated it