The price of power
Complexity has a price, and it's paid in mistakes. The more powerful the tool, the larger the damage a single error can do, so the second half of "why we do this" is "why we do it carefully".
BGP shows this most clearly, because it connects you to everyone. Two well-known incidents:
- 2008: an operator meant to block a video site inside its own country by announcing a more specific route for the site's addresses. The announcement leaked to its upstream provider, which didn't filter it, and from there across much of the internet. Traffic for that site worldwide was drawn towards a network that dropped it, for around two hours.
- 2021: during routine maintenance, a large social network's backbone was disconnected by mistake. Its DNS servers, seeing the backbone unreachable, withdrew their BGP announcements as designed, and the company's services vanished from the internet for hours. Its own engineers lost the tools they'd normally use to fix it.
Neither was caused by BGP being "too complicated". Both were caused by a powerful system without enough guard rails. So the lessons pair every protocol with its guard rails:
- Filter both ways. Only accept what a neighbour should send you; only announce what's yours. (Setting up VyOS with basic BGP)
- Check who owns what. Routing registries and RPKI let networks reject announcements the address holder never authorised. (IRR, RPSL and AS-SETs, RPKI and Route Origin Validation)
- Make changes reversible. Commit with an automatic rollback, keep configuration history off the router. (Operating VyOS)
- Know your normal, and watch it. Prefix counts, session states, logs. (Debugging BGP)
- Test in a lab first. Every configuration on this site was run in a lab before it was written down, and you can do the same. (Building a network lab on an Apple Silicon Mac)
Complexity you can't operate safely is worse than simplicity. The goal is a network that's as simple as its job allows, and no simpler.
Check your understanding
What's the main lesson of the big BGP incidents?
- Powerful tools need guard rails: filters, origin validation, reversible changes and monitoring.
- BGP is too dangerous and should be replaced by static routing.
- Only large companies make these mistakes.
- Use longer timers so changes spread more slowly.