A session on interface addresses
Start with the obvious design: iBGP between the link A addresses. It comes up in seconds, R1 learns 203.0.113.0/24 with next hop 192.0.2.1, and everything works.
Now break link A.
For the first minutes, nothing looks wrong. R1's eth0 is still up, the session still says Established, and the route is still installed pointing at link A. But traffic to 203.0.113.0/24 disappears into the dead link. BGP only finds out when no keepalive has arrived for the hold time: 180 seconds by default on VyOS. Until then, R1 is blackholing traffic while every status command says it's fine.
Then it stays down, even though there's a perfectly good second link. When the hold timer fires, R1 tears the session down and withdraws the route. Link B works (R1 can ping R2 across it), but the session's addresses belong to link A, so there's nothing to reconnect to. R1 has lost R2's prefix until someone fixes link A.
Even R2, which saw its own interface go down, waited for the hold timer. FRR's fast reaction to an interface going down (fast-external-failover) only applies to directly connected eBGP sessions, not iBGP.
You could add a second session over link B. That doubles the sessions, policy and monitoring, and makes BGP choose between two copies of every route instead of using both links. Loopbacks are the cleaner answer.
set protocols bgp system-as '64500'
set protocols bgp parameters router-id '198.51.100.1'
set protocols bgp neighbor 192.0.2.1 remote-as 'internal'
set protocols bgp neighbor 192.0.2.1 description 'R2 via link A'
set protocols bgp neighbor 192.0.2.1 address-family ipv4-unicast- Lines 3–5: The neighbor is R2's address on link A, so the session lives and dies with that link.
set protocols bgp system-as '64500'
set protocols bgp parameters router-id '198.51.100.2'
set protocols bgp address-family ipv4-unicast network 203.0.113.0/24
set protocols bgp neighbor 192.0.2.0 remote-as 'internal'
set protocols bgp neighbor 192.0.2.0 description 'R1 via link A'
set protocols bgp neighbor 192.0.2.0 address-family ipv4-unicast- Line 3: Originate 203.0.113.0/24.
- Lines 4–6: R1's link A address.
vyos@R1:~$ show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 198.51.100.1, local AS number 64500 VRF default vrf-id 0
BGP table version 1
RIB entries 1, using 160 bytes of memory
Peers 1, using 30 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.0.2.1 4 64500 4 3 1 0 0 00:00:15 1 0 R2 via link A
Total number of neighbors 1
vyos@R1:~$ show ip route 203.0.113.0/24
Routing entry for 203.0.113.0/24
Known via "bgp", distance 200, metric 0, best
Last update 00:00:15 ago
Flags: Recursion iBGP Selected
Status: Installed
* 192.0.2.1, via eth0, weight 1
vyos@R1:~$ ping 203.0.113.1 count 3
PING 203.0.113.1 (203.0.113.1) 56(84) bytes of data.
64 bytes from 203.0.113.1: icmp_seq=1 ttl=64 time=0.801 ms
64 bytes from 203.0.113.1: icmp_seq=2 ttl=64 time=2.96 ms
64 bytes from 203.0.113.1: icmp_seq=3 ttl=64 time=2.80 ms
--- 203.0.113.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2008ms
rtt min/avg/max/mdev = 0.801/2.187/2.959/0.982 ms- Line 10: Established, one prefix received.
- Line 19: Via link A only. Link B carries nothing.
- Lines 23–25: R2's prefix answers.
vyos@R1:~$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address MAC VRF MTU S/L Description
----------- --------------- ----------------- ------- ----- ----- -------------
eth0 192.0.2.0/31 52:54:00:01:00:01 default 1500 u/u LINK-A-TO-R2
eth1 192.0.2.2/31 52:54:00:01:01:01 default 1500 u/u LINK-B-TO-R2
lo 127.0.0.1/8 00:00:00:00:00:00 default 65536 u/u
198.51.100.1/32
::1/128
vyos@R1:~$ show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 198.51.100.1, local AS number 64500 VRF default vrf-id 0
BGP table version 1
RIB entries 1, using 160 bytes of memory
Peers 1, using 30 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.0.2.1 4 64500 4 3 1 0 0 00:00:33 1 0 R2 via link A
Total number of neighbors 1
vyos@R1:~$ show ip bgp neighbors 192.0.2.1 | match "BGP state|Last read|Hold time is"
BGP state = Established, up for 00:00:34
Last read 00:00:33, Last write 00:00:33
Hold time is 180 seconds, keepalive interval is 60 seconds
vyos@R1:~$ ping 203.0.113.1 count 3
PING 203.0.113.1 (203.0.113.1) 56(84) bytes of data.
--- 203.0.113.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2063ms- Line 5: R1's eth0 is still up: it can't see the failure.
- Line 19: The session still says Established…
- Lines 24–25: …because the last keepalive was only 33 seconds ago. It will wait for the full 180-second hold time.
- Line 30: Meanwhile, traffic to R2's prefix is lost.
vyos@R1:~$ show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 198.51.100.1, local AS number 64500 VRF default vrf-id 0
BGP table version 2
RIB entries 0, using 0 bytes of memory
Peers 1, using 30 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.0.2.1 4 64500 4 7 0 0 0 00:00:08 Active 0 R2 via link A
Total number of neighbors 1
vyos@R1:~$ show ip bgp neighbors 192.0.2.1 | match "BGP state|Last reset|Hold Timer"
BGP state = Active
Last reset 00:00:08, Notification sent (Hold Timer Expired)
vyos@R1:~$ show ip route 203.0.113.0/24
% Network not in table
vyos@R1:~$ ping 192.0.2.3 count 2
PING 192.0.2.3 (192.0.2.3) 56(84) bytes of data.
64 bytes from 192.0.2.3: icmp_seq=1 ttl=64 time=5.55 ms
64 bytes from 192.0.2.3: icmp_seq=2 ttl=64 time=2.88 ms
--- 192.0.2.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1006ms
rtt min/avg/max/mdev = 2.883/4.215/5.548/1.332 ms- Line 10: Active: trying to reconnect to 192.0.2.1, which is on the dead link.
- Line 15: Torn down by the hold timer, about two and a half minutes after the failure.
- Line 17: The route to R2's prefix is gone…
- Lines 20–25: …although link B works perfectly.
vyos@R2:~$ show ip bgp neighbors 192.0.2.0 | match "BGP state|Last reset"
BGP state = Active
Last reset 00:00:17, Notification sent (Hold Timer Expired)- Line 3: R2 saw its own interface go down, but iBGP still waited for the hold timer.
Check your understanding
After link A fails silently, how long does R1 keep forwarding traffic into the dead link?
- Not at all; BGP notices immediately
- Up to the hold time, 180 seconds by default
- Until link A is repaired
- 40 seconds, the OSPF dead interval