frr.conf and vtysh
FRR keeps its whole configuration in one file, /etc/frr/frr.conf, in the same syntax you type at its command line. vtysh is that command line: it connects to every running daemon at once, so show bgp summary, show ip ospf neighbor and configure terminal all work from one shell. Anything you'd type in vtysh can go in the file, and the other way round.
The edge router's file has five parts:
- Housekeeping: the hostname, logging to syslog (so
journalctl -u frrshows it), and one integrated file for all daemons. - A blackhole route for the office's /24. BGP only announces a prefix that exists in the routing table. The real routes are more specific (the /26 behind the core), so a blackhole for the whole /24 makes it exist, and drops traffic for any part of it that isn't in use instead of sending it back to the ISP.
- Prefix lists and route maps decide what's accepted and announced; the next section looks at them closely.
- The BGP sessions: one per ISP, each with an inbound and an outbound route map, a cap on the number of prefixes, and soft-reconfiguration so you can see what was received before filtering.
- OSPF on the inside link, with
default-information originate: as long asedgehas a default route (from BGP), it offers one to the core.
Starting it
Turn on bgpd and ospfd in /etc/frr/daemons, install the configuration (owned by the frr user, not world-readable), and restart the service. watchfrr now also runs bgpd and ospfd. A few seconds later both sessions are Established: each ISP's line shows a count of prefixes received (after filtering) and sent. With bgp log-neighbor-changes and log-adjacency-changes, the journal records each session and adjacency coming up, which is where you'll look first when one goes down.
! /etc/frr/frr.conf on edge: two upstreams by BGP, the office core by OSPF.
frr defaults traditional
hostname edge
log syslog informational
service integrated-vtysh-config
!
! The office's prefix exists as a route here, so BGP can announce it.
ip route 203.0.113.0/24 blackhole
!
ip prefix-list OUR-PREFIX seq 5 permit 203.0.113.0/24
!
ip prefix-list FROM-UPSTREAM seq 5 deny 203.0.113.0/24 le 32
ip prefix-list FROM-UPSTREAM seq 10 deny 10.0.0.0/8 le 32
ip prefix-list FROM-UPSTREAM seq 15 deny 100.64.0.0/10 le 32
ip prefix-list FROM-UPSTREAM seq 20 deny 172.16.0.0/12 le 32
ip prefix-list FROM-UPSTREAM seq 25 deny 192.168.0.0/16 le 32
ip prefix-list FROM-UPSTREAM seq 30 deny 0.0.0.0/0 ge 25
ip prefix-list FROM-UPSTREAM seq 100 permit 0.0.0.0/0 le 24
!
route-map ISP1-IN permit 10
match ip address prefix-list FROM-UPSTREAM
set local-preference 200
exit
!
route-map ISP2-IN permit 10
match ip address prefix-list FROM-UPSTREAM
exit
!
route-map ISP1-OUT permit 10
match ip address prefix-list OUR-PREFIX
exit
!
route-map ISP2-OUT permit 10
match ip address prefix-list OUR-PREFIX
set as-path prepend 64500 64500
exit
!
interface eth2
ip ospf area 0
ip ospf network point-to-point
exit
!
router bgp 64500
bgp router-id 10.255.0.1
bgp log-neighbor-changes
neighbor 192.0.2.0 remote-as 64501
neighbor 192.0.2.0 description isp1 (primary)
neighbor 192.0.2.2 remote-as 64502
neighbor 192.0.2.2 description isp2 (backup)
!
address-family ipv4 unicast
network 203.0.113.0/24
neighbor 192.0.2.0 soft-reconfiguration inbound
neighbor 192.0.2.0 route-map ISP1-IN in
neighbor 192.0.2.0 route-map ISP1-OUT out
neighbor 192.0.2.0 maximum-prefix 1000
neighbor 192.0.2.2 soft-reconfiguration inbound
neighbor 192.0.2.2 route-map ISP2-IN in
neighbor 192.0.2.2 route-map ISP2-OUT out
neighbor 192.0.2.2 maximum-prefix 1000
exit-address-family
exit
!
router ospf
ospf router-id 10.255.0.1
log-adjacency-changes
default-information originate
exit
!- Lines 2–5: Housekeeping: the default timers profile, the name, logging to syslog (and so the journal), one file for all daemons.
- Line 8: The office's whole /24 as a blackhole route, so BGP has something to announce.
- Lines 10–18: The prefix lists: what the office owns, and what it accepts from the ISPs.
- Lines 20–36: One route map per direction per ISP. Next section.
- Lines 38–41: OSPF runs on the inside interface only.
- Lines 43–49: Two eBGP neighbours, each with a description that shows up in every
showcommand. - Lines 51–60: The office prefix, and each neighbour's policy: soft-reconfiguration, route maps in and out, a prefix limit.
- Lines 64–67: OSPF: log adjacency changes, and offer the core a default route while we have one.
root@edge:~# sed -i "s/^bgpd=no/bgpd=yes/; s/^ospfd=no/ospfd=yes/" /etc/frr/daemons && grep -E "^(bgpd|ospfd)=" /etc/frr/daemons
bgpd=yes
ospfd=yes
root@edge:~# install -m 640 -o frr -g frr frr.conf /etc/frr/frr.conf && systemctl restart frr && systemctl status frr --no-pager | sed -n "/CGroup/,/^$/p"
CGroup: /docker/68dabd70d05c7d5877f2dde6ce7bac3e44adce867fa60f2035d700237fbdaeae/system.slice/frr.service
├─365 /usr/lib/frr/watchfrr -d -F traditional zebra mgmtd bgpd ospfd staticd
├─377 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000
├─382 /usr/lib/frr/mgmtd -d -F traditional -A 127.0.0.1
├─384 /usr/lib/frr/bgpd -d -F traditional -A 127.0.0.1
├─391 /usr/lib/frr/ospfd -d -F traditional -A 127.0.0.1
└─394 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1
root@edge:~# vtysh -c "show bgp summary"
IPv4 Unicast Summary:
BGP router identifier 10.255.0.1, local AS number 64500 VRF default vrf-id 0
BGP table version 3
RIB entries 6, using 768 bytes of memory
Peers 2, using 47 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
192.0.2.0 4 64501 6 4 3 0 0 00:00:11 2 1 isp1 (primary)
192.0.2.2 4 64502 7 4 3 0 0 00:00:11 2 1 isp2 (backup)
Total number of neighbors 2
root@edge:~# journalctl -u frr --no-pager -o cat | grep -E "ADJCHANGE|Up|Full" | tail -4
[YWPB2-VEAQY] ASBR[default:Status:1]: Update
[N9HHH-F8H1M] %ADJCHANGE: neighbor 192.0.2.0(isp1) in vrf default Up
[N9HHH-F8H1M] %ADJCHANGE: neighbor 192.0.2.2(isp2) in vrf default Up
[Y05P2-YJVXY] AdjChg: Nbr 10.255.0.2, NbrIP 10.0.0.2 (default) on eth2:10.0.0.1: Loading -> Full (LoadingDone)- Lines 1–3: Turn on the two protocol daemons.
- Lines 5–12: After a restart, watchfrr also runs bgpd and ospfd.
- Lines 23–25: Both sessions Established: two routes accepted from each ISP, one (the office /24) sent to each.
- Lines 29–33: The journal records each session and the OSPF adjacency coming up.
Check your understanding
Why does edge's frr.conf have ip route 203.0.113.0/24 blackhole?
- BGP's
networkstatement only announces a prefix that exists in the routing table. - To stop the ISPs sending traffic to the office.
- OSPF needs it to reach the core.
- To drop spoofed packets.