Recently I decided to deepen my knowledge of BGP and MPLS, so I started studying for CCIP certification, which cover both arguments. As suggested by many people I started with BGP “Internet Routing Architectures 2nd Edition”, by Sam Halabi, Cisco Press. It seems to be a really good book, and It contains a lot of scenarios, so I have though to build some GNS3 labs inspired by them. The first interesting one is on chapter 7, Scenario 2, and I tried to build a lab on it.
EDIT 2010-09-07: thanks to a friend, Akuavi Dossou, I found an error in the previous .ZIP file. Cust1_SF and Cust1_NY configurations in the “Base” directory were wrong, 192.168.3.0/30 IP addresses were assigned to the S0/1 interfaces instead of E1/1 and iBGP between the routers didn’t come up. Configurations in the “Final” directory were OK. I fixed the error and uploaded the new .ZIP file. Sorry for the unexpected troubleshooting exercise. 🙂
BGP - Multihoming to a Single Provider with partial routing
We have a customer with 2 routers multihomed to a single provider: customer’s SF router is connected to provider’s SF router, customer NY router to provider NY router. Customer is in AS 65001, provider in AS 65000. Both customer and provider have an internal link connecting their routers. Provider has some other customers directly connected to its routers, and it’s connected to a NAP through the NY router.
We want our customer to have an optimal routing table to reach both provider’s customers and the full Internet.
We start configuring BGP on the routers; provider’s routers are already configured for iBGP, so we just have to enable the customer neighbors:
Provider_SF(config)#router bgp 65000
Provider_SF(config-router)#neighbor 192.168.1.2 remote-as 65001
Provider_NY(config)#router bgp 65000
Provider_NY(config-router)#neighbor 192.168.2.2 remote-as 65001
Customer’s routers need to be configured for both iBGP and eBGP:
Cust1_SF(config)#router bgp 65001
Cust1_SF(config-router)#network 192.168.10.0
Cust1_SF(config-router)#neighbor 192.168.1.1 remote-as 65000
Cust1_SF(config-router)#neighbor 192.168.3.2 remote-as 65001
Cust1_SF(config-router)#neighbor 192.168.3.2 next-hop-self
Cust1_NY(config)#router bgp 65001
Cust1_NY(config-router)#network 192.168.20.0
Cust1_NY(config-router)#neighbor 192.168.2.1 remote-as 65000
Cust1_NY(config-router)#neighbor 192.168.3.1 remote-as 65001
Cust1_NY(config-router)#neighbor 192.168.3.1 next-hop-self
As soon as BGP sessions come up and are stable, we can take a look at the customer’s routing table:
Cust1_SF#show ip route bgp
1.0.0.0/24 is subnetted, 1 subnets
B 1.1.1.0 [20/0] via 192.168.1.1, 00:27:53
B 192.168.20.0/24 [200/0] via 192.168.3.2, 00:19:03
10.0.0.0/24 is subnetted, 4 subnets
B 10.2.1.0 [20/0] via 192.168.1.1, 00:27:53
B 10.1.2.0 [20/0] via 192.168.1.1, 00:27:53
B 10.2.2.0 [20/0] via 192.168.1.1, 00:27:53
B 10.1.1.0 [20/0] via 192.168.1.1, 00:27:53
Cust1_NY#show ip route bgp
1.0.0.0/24 is subnetted, 1 subnets
B 1.1.1.0 [20/0] via 192.168.2.1, 00:21:18
B 192.168.10.0/24 [200/0] via 192.168.3.1, 00:19:29
10.0.0.0/24 is subnetted, 4 subnets
B 10.2.1.0 [20/0] via 192.168.2.1, 00:21:18
B 10.1.2.0 [20/0] via 192.168.2.1, 00:21:18
B 10.2.2.0 [20/0] via 192.168.2.1, 00:21:18
B 10.1.1.0 [20/0] via 192.168.2.1, 00:21:18
As we can see, all routes are in the table, but an half of them is using a suboptimal path: remote subnets are reached via the local BGP peer, and this is not good for those directly connected to the other peer. For example, SF customer router should reach C4 and C5 via the internal path to NY router, and then through the Customer_NY-Provider_NY link.
Let’s take a look at the provider’s routing table too:
Provider_SF#sh ip route bgp
1.0.0.0/24 is subnetted, 1 subnets
B 1.1.1.0 [200/0] via 172.16.0.2, 00:01:59
B 192.168.10.0/24 [20/0] via 192.168.1.2, 00:02:03
B 192.168.20.0/24 [20/0] via 192.168.1.2, 00:01:32
10.0.0.0/24 is subnetted, 4 subnets
B 10.2.1.0 [200/0] via 172.16.0.2, 00:01:59
B 10.2.2.0 [200/0] via 172.16.0.2, 00:01:59
Provider_NY#sh ip route bgp
B 192.168.10.0/24 [20/0] via 192.168.2.2, 00:02:59
B 192.168.20.0/24 [20/0] via 192.168.2.2, 00:02:59
10.0.0.0/24 is subnetted, 4 subnets
B 10.1.2.0 [200/0] via 172.16.0.1, 00:02:59
B 10.1.1.0 [200/0] via 172.16.0.1, 00:02:59
Suboptimal paths exist here too: customer’s routes are always reached via the local BGP peer.
Why does BGP prefer routes from the local peer? Let’s get more details about one of these routes:
Cust1_SF#sh ip bgp 10.1.1.0
BGP routing table entry for 10.1.1.0/24, version 2
Paths: (2 available, best #2, table Default-IP-Routing-Table)
Advertised to update-groups:
1
65000
192.168.3.2 from 192.168.3.2 (192.168.20.1)
Origin IGP, metric 0, localpref 100, valid, internal
65000
192.168.1.1 from 192.168.1.1 (192.168.1.1)
Origin IGP, metric 0, localpref 100, valid, external, best
Cust1_SF receives C2 subnet (10.1.1.0/24) from Provider_SF (eBGP) and from Cust1_NY (iBGP). Both routes have the same value for all attributes, the only difference is the way they have been learnt: internal (iBGP) and external (eBGP). The BGP decision process picks the external one: this is the rule.
Now it’s time to build an optimal routing table; we want to reach C2 and C3 subnets through SF provider’s router, C4 and C5 through NY router, and unknown destinations (1.1.1.0/24) via NY router too.
On the customer’s side we can achieve this objective by setting different LocalPref values on prefixes learnt at the AS border: SF router will set C2 and C3 with LocalPref = 300, other prefixes with LocalPref = 200; NY router will set C4 and C5 with LocalPref = 300, other prefixes with 250. We can do this using route-maps:
Cust1_SF:
ip prefix-list C2 seq 5 permit 10.1.1.0/24
ip prefix-list C3 seq 5 permit 10.1.2.0/24
!
route-map EBGP-With-ProviderSF_IN permit 10
match ip address prefix-list C2 C3
set local-preference 300
route-map EBGP-With-ProviderSF_IN permit 20
set local-preference 200
!
router bgp 65001
neighbor 192.168.1.1 route-map EBGP-With-ProviderSF_IN in
Cust1_NY:
ip prefix-list C4 seq 5 permit 10.2.1.0/24
ip prefix-list C5 seq 5 permit 10.2.2.0/24
!
route-map EBGP-With-ProviderNY_IN permit 10
match ip address prefix-list C4 C5
set local-preference 300
route-map EBGP-With-ProviderNY_IN permit 20
set local-preference 250
!
router bgp 65001
neighbor 192.168.2.1 route-map EBGP-With-ProviderNY_IN in
A clear ip bgp 65000 command forces BGP to update routes and, now, they are filtered by the route-map. This is the result:
Cust1_SF#show ip route bgp
1.0.0.0/24 is subnetted, 1 subnets
B 1.1.1.0 [200/0] via 192.168.3.2, 00:10:16
B 192.168.20.0/24 [200/0] via 192.168.3.2, 00:10:45
10.0.0.0/24 is subnetted, 4 subnets
B 10.2.1.0 [200/0] via 192.168.3.2, 00:10:16
B 10.1.2.0 [20/0] via 192.168.1.1, 00:18:32
B 10.2.2.0 [200/0] via 192.168.3.2, 00:10:16
B 10.1.1.0 [20/0] via 192.168.1.1, 00:18:32
Cust1_NY#show ip route bgp
1.0.0.0/24 is subnetted, 1 subnets
B 1.1.1.0 [20/0] via 192.168.2.1, 00:10:32
B 192.168.10.0/24 [200/0] via 192.168.3.1, 00:11:01
10.0.0.0/24 is subnetted, 4 subnets
B 10.2.1.0 [20/0] via 192.168.2.1, 00:10:32
B 10.1.2.0 [200/0] via 192.168.3.1, 00:11:01
B 10.2.2.0 [20/0] via 192.168.2.1, 00:10:32
B 10.1.1.0 [200/0] via 192.168.3.1, 00:11:01
We have wath we needed! But what’s up on the Provider’s side?
Provider_SF#sh ip bgp 192.168.10.0
BGP routing table entry for 192.168.10.0/24, version 21
Paths: (2 available, best #2, table Default-IP-Routing-Table)
Advertised to update-groups:
2
65001
172.16.0.2 from 172.16.0.2 (192.168.2.1)
Origin IGP, metric 0, localpref 100, valid, internal
65001
192.168.1.2 from 192.168.1.2 (192.168.10.1)
Origin IGP, metric 0, localpref 100, valid, external, best
Nothing! Each router still prefers local BGP peer to reach customer’s subnets. Our route-maps just set LocalPref on customer’s routers, so their influence is limited to just the way learnt routes are preferred each other. To let the provider use an optimal routing table we can set the MED attribute on outgoing BGP UPDATEs, so that locally connected subnets could have a better (lower) metric than others. As usual, we can do that using route-maps and prefix lists:
Cust1_SF:
ip prefix-list Net10 seq 5 permit 192.168.10.0/24
!
route-map EBGP-With-ProviderSF_OUT permit 10
match ip address prefix-list Net10
set metric 200
route-map EBGP-With-ProviderSF_OUT permit 20
set metric 300
!
router bgp 65001
neighbor 192.168.1.1 route-map EBGP-With-ProviderSF_OUT out
Cust1_NY:
ip prefix-list Net20 seq 5 permit 192.168.20.0/24
!
route-map EBGP-With-ProviderNY_OUT permit 10
match ip address prefix-list Net20
set metric 200
route-map EBGP-With-ProviderNY_OUT permit 20
set metric 250
!
router bgp 65001
neighbor 192.168.2.1 route-map EBGP-With-ProviderNY_OUT out
Then:
Cust1_SF#clear ip bgp 65000
Cust1_NY#clear ip bgp 65000
Wait until BGP does its job, then:
Provider_SF#sh ip route bgp | inc 192
B 192.168.10.0/24 [20/200] via 192.168.1.2, 00:06:05
B 192.168.20.0/24 [200/200] via 172.16.0.2, 00:06:04
Provider_NY#sh ip route bgp | inc 192
B 192.168.10.0/24 [200/200] via 172.16.0.1, 00:06:46
B 192.168.20.0/24 [20/200] via 192.168.2.2, 00:06:46
Now our provider reaches our subnets in the way we need.
Our subnets are still received from the local eBGP peer, but with different metrics:
Provider_SF#sh ip bgp | inc 192.168
BGP table version is 29, local router ID is 192.168.1.1
*> 192.168.10.0 192.168.1.2 200 0 65001 i
*>i192.168.20.0 172.16.0.2 200 100 0 65001 i
* 192.168.1.2 300 0 65001 i
192.168.10.0 is the subnet locally connected to the provider’s router peer, so it has a low metric (200), but 192.168.20.0 is not, so it has a higher metric (300). Provider’s router prefers to reach that subnet via the internal link through the other iBGP peer, which receives the subnet with a lower metric (200) from it’s eBGP peer.
On the lab .ZIP file you can find two configuration sets: base and final. The first contains just a base configuration with IP addresses and iBGP on the provider’s side. The final configuration contains all commands reported in this post.
EDIT 2010-09-07: thanks to a friend, Akuavi Dossou, I found an error in the previous .ZIP file. Cust1_SF and Cust1_NY configurations in the “Base” directory were wrong, 192.168.3.0/30 IP addresses were assigned to the S0/1 interfaces instead of E1/1 and iBGP between the routers didn’t come up. Configurations in the “Final” directory were OK. I fixed the error and uploaded the new .ZIP file. Sorry for the unexpected troubleshooting exercise. 🙂
Download the Lab here.
