Now, more than ever, we need a more resilient Internet.
This is the first sentence that shows up on the website of MANRS, a global project that aims to improve the security of Internet routing. Every network is in a good position to actively pursue such an important goal, and MANRS helps them by offering different programmes with guidelines and suggestions based on industry best practices, focused on the role of each network: ISPs, CDNs, Internet Exchange Points (IXPs).
In this post I’ll focus on the IXP programme, and I’ll share some thoughts on how ARouteServer, a tool I’ve been working on for several years, can help IXP operators to easily deploy and operate secure route servers and meet MANRS requirements.
Role of Internet Exchanges in the routing security landscape
Networks of any size interconnect each other on the exchange platforms across the world, to shorten the path between their mutual customers (and hence improve latency), to increase the availability of a path between them, to remove intermediary networks, to keep local traffic local, to lower operating costs and many more reasons.
Route servers represent an important facility that IXPs offer to their participants. They allow members to benefit from a quick return of investment, by speeding up the growth of networks they can peer with since the beginning of their membership, and also to lower the burden of maintaining a potentially high number of bilateral BGP sessions.
But with shorter paths and closer adjacencies comes a higher chance of being impacted by a routing incident. Indeed, routes learned over the IX peering fabrics may have the shortest AS_PATH possible, and hence be preferred over others. As a result, a BGP misconfiguration on one side could adversely impair the reachability of even large portions of the Internet on the peer network, if it leads to announcing broken routes. Regardless of whether the peering relationship is a bilateral one or one established through a route server.
This is where IXPs play an active role in the Internet routing security landscape: a secure route server configuration that features well-established industry best practices and solutions helps certainly to minimize (or even completely prevent) routing incidents.
On the other hand, a facility that theoretically should add value to an IXP and to their members could easily become a weakness and a reason for shame if it propagates bogus announcements, helping to increase the blast radius of a potential incident.
That’s why it’s important for IXP to demonstrate their commitment to implement and promote routing hygiene best practices. Something that they can do for example by joining the MANRS IXP programme.
The IXP programme requirements
In essence, the IXP programme is a set of 5 action items; to be eligible for joining MANRS an IXP must implement at least 3 of them: 2 are pre-selected and the third one can be chosen among the remaining 3 open-choice items.
Route filtering at the route servers
The first one of the pre-selected mandatory action items is “Prevent propagation of incorrect routing information”, which is focused on route filtering at the route servers. And it is also the one ARouteServer can strongly help on.
The programme action item wants an IX operator to implement IRR- or RPKI-based filtering along with some additional routing hygiene best-practices (like dropping routes towards bogon prefixes or those containing well-known undesired ASNs in the AS_PATH).
Meeting this requirement could represent a significant effort if the IX operator wanted to build a home-grown configuration for the route servers. Also, keeping the configuration up to date with the ever-changing policies of the IX participants (ROAs, IRR data, max-prefix limits) would require constant attention and timely commitment, that would become more and more time consuming as the exchange participant list grows.
Conversely, a consolidated tool like ARouteServer could make it trivial and less prone to errors. Let’s see how.
The project started in Jan 2017 and it’s now used by ~20 different IXPs across the globe, on both BIRD and OpenBGPD route servers. Being backed by the OSS community makes contributions and bug-fixes faster, and also it allows an easy consolidation of industry best-practices into a single piece of software.
A very simple installation process (based on Python3 in the example below) allows the tool to be up&running in a few minutes.
apt-get install bgpq3 # to fetch data from IRRs python3 -m venv ~/.virtualenvs/arouteserver source ~/.virtualenvs/arouteserver/bin/activate pip install arouteserver arouteserver setup
The arouteserver configure command will then guide the operator through some basic questions about the route server environment (ASN, router ID, local subnets) and eventually it will build a policy definition file, based on best practices and suggestions, which will include, among other features: IRR-based route filtering, RPKI BGP Origin Validation, PeeringDB integration to fetch up to date information from network records, a rich set of BGP communities schema. (Full list of features is available on the doc site.)
At this point, it’s just a matter of defining the route server clients in a YAML file…
clients:
- asn: 64496
ip: "192.0.2.11"
- asn: 64511
ip:
- "192.0.2.22"
- "2001:db8:1:1::22"
… (or to import pre-existing clients from several sources: IXP Manager, Euro-IX JSON Export files, PeeringDB) and then running commands like these:
# for BIRD: arouteserver bird --ip-ver 4 -o /etc/bird/bird4.conf # for OpenBGPD: arouteserver openbgpd -o /etc/bgpd.conf
The tool will automatically gather information such as IRR data, RPKI ROAs, max-prefix limits and eventually it will build the configuration file that will be ready to be loaded into the selected BGP daemon (see some examples). All the filters required by the MANRS IXP programme will be automatically in place. Also, it will be trivial to keep the information up to date: just a matter of running the command again (maybe via a cronjob-like script) and then reload the configuration in the BGP daemon.
Looking glass and assistance to keep accurate routing information
ARouteServer is also able to help on two more action items of the IXP programme.
One of the main goals of MANRS is to spread the concept of a healthy routing environment and to help network operators to run a safer Internet. The second mandatory action item from the programme wants IXPs to promote good routing manners to their membership, and one among the proposed ways to do it is by offering them assistance to maintain accurate routing information in either or both an IRR registry or the RPKI ecosystem.
Reading further down through the programme, we can see that one of the 3 open-choice action items that an aspiring MANRS member can choose to implement is about providing a looking glass. Such tools demonstrate to be very useful both for IXP members and also for external network operators, to debug routing anomalies and to help troubleshooting during routing incidents.
Both requirements are something that ARouteServer can help with.
The route server configuration generated by the tool uses BGP communities to tag routes that are rejected and to classify them according to the reject reason for which they were considered as not valid (RPKI INVALID, Invalid ASN in AS_PATH, …). Also, the configuration that it builds allows to integrate alice-lg (a BIRD looking glass) out of the box. The combination of both represents a powerful method to assist IXP members in detecting, troubleshooting and fixing unhealthy routes that are announced to the route server. Using a looking glass it will be easy to understand why a route was rejected by the route server, and consequently to help the member to fix it.
Additionally, a side tool that ships with ARouteServer (Invalid routes reporter) can also be deployed to automatically report rejected routes to the IXP members that announce them, along with a description about why they were dropped. Just a few minutes (or even seconds) after the unhealthy route is received by the server and the member’s NOC can be aware of it.
Conclusions
The MANRS programmes are very good readings that help everyone to be conscious of how a good Internet actor should operate and behave for the sake of a more robust routing ecosystem. I would strongly suggest any IXP to take a look at the whole document, since it contains many more interesting ideas and proposals than those that I’ve mentioned in this post.
The value proposition about operating a secure route server is straightforward (not mentioning the terrifying idea of having a red label that says “insecure” on the https://peering.exposed/ spreadsheet!).
ARouteServer can help! But another software that can help too and definitely worth a mention is IXP Manager, a huge framework that offers many tools to deal with all the aspects related to the IXP management, including MANRS compliance.
What should be done is clear; how to do it is also made easier thanks to consolidated automation-based tools. It’s just time to do it 😉
Latest posts by Pier Carlo Chiodi (see all)
- Good MANRS for IXPs route servers made easier - 11 December 2020
- Route server feature-rich and automatic configuration - 13 February 2017
- Large BGP Communities playground - 15 September 2016
Leave a Reply