GNS3 Lab: BGP – Multihoming to a Single Provider with partial routing

Recently I decided to deepen my knowledge of BGP and MPLS, so I started studying for CCIP certification, which cover both arguments. As suggested by many people I started with BGP “Internet Routing Architectures 2nd Edition”, by Sam Halabi, Cisco Press. It seems to be a really good book, and It contains a lot of scenarios, so I have though to build some GNS3 labs inspired by them. The first interesting one is on chapter 7, Scenario 2, and I tried to build a lab on it.

EDIT 2010-09-07: thanks to a friend, Akuavi Dossou, I found an error in the previous .ZIP file. Cust1_SF and Cust1_NY configurations in the “Base” directory were wrong, 192.168.3.0/30 IP addresses were assigned to the S0/1 interfaces instead of E1/1 and iBGP between the routers didn’t come up. Configurations in the “Final” directory were OK. I fixed the error and uploaded the new .ZIP file. Sorry for the unexpected troubleshooting exercise. 🙂

BGP - Multihoming to a Single Provider with partial routing

BGP - Multihoming to a Single Provider with partial routing

We have a customer with 2 routers multihomed to a single provider: customer’s SF router is connected to provider’s SF router, customer NY router to provider NY router. Customer is in AS 65001, provider in AS 65000. Both customer and provider have an internal link connecting their routers. Provider has some other customers directly connected to its routers, and it’s connected to a NAP through the NY router.

We want our customer to have an optimal routing table to reach both provider’s customers and the full Internet.

We start configuring BGP on the routers; provider’s routers are already configured for iBGP, so we just have to enable the customer neighbors:

Provider_SF(config)#router bgp 65000
Provider_SF(config-router)#neighbor 192.168.1.2 remote-as 65001
Provider_NY(config)#router bgp 65000
Provider_NY(config-router)#neighbor 192.168.2.2 remote-as 65001

Customer’s routers need to be configured for both iBGP and eBGP:

Cust1_SF(config)#router bgp 65001
Cust1_SF(config-router)#network 192.168.10.0
Cust1_SF(config-router)#neighbor 192.168.1.1 remote-as 65000
Cust1_SF(config-router)#neighbor 192.168.3.2 remote-as 65001
Cust1_SF(config-router)#neighbor 192.168.3.2 next-hop-self
Cust1_NY(config)#router bgp 65001
Cust1_NY(config-router)#network 192.168.20.0
Cust1_NY(config-router)#neighbor 192.168.2.1 remote-as 65000
Cust1_NY(config-router)#neighbor 192.168.3.1 remote-as 65001
Cust1_NY(config-router)#neighbor 192.168.3.1 next-hop-self

As soon as BGP sessions come up and are stable, we can take a look at the customer’s routing table:

Cust1_SF#show ip route bgp
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [20/0] via 192.168.1.1, 00:27:53
B    192.168.20.0/24 [200/0] via 192.168.3.2, 00:19:03
     10.0.0.0/24 is subnetted, 4 subnets
B       10.2.1.0 [20/0] via 192.168.1.1, 00:27:53
B       10.1.2.0 [20/0] via 192.168.1.1, 00:27:53
B       10.2.2.0 [20/0] via 192.168.1.1, 00:27:53
B       10.1.1.0 [20/0] via 192.168.1.1, 00:27:53
Cust1_NY#show ip route bgp
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [20/0] via 192.168.2.1, 00:21:18
B    192.168.10.0/24 [200/0] via 192.168.3.1, 00:19:29
     10.0.0.0/24 is subnetted, 4 subnets
B       10.2.1.0 [20/0] via 192.168.2.1, 00:21:18
B       10.1.2.0 [20/0] via 192.168.2.1, 00:21:18
B       10.2.2.0 [20/0] via 192.168.2.1, 00:21:18
B       10.1.1.0 [20/0] via 192.168.2.1, 00:21:18

As we can see, all routes are in the table, but an half of them is using a suboptimal path: remote subnets are reached via the local BGP peer, and this is not good for those directly connected to the other peer. For example, SF customer router should reach C4 and C5 via the internal path to NY router, and then through the Customer_NY-Provider_NY link.

Let’s take a look at the provider’s routing table too:

Provider_SF#sh ip route bgp
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [200/0] via 172.16.0.2, 00:01:59
B    192.168.10.0/24 [20/0] via 192.168.1.2, 00:02:03
B    192.168.20.0/24 [20/0] via 192.168.1.2, 00:01:32
     10.0.0.0/24 is subnetted, 4 subnets
B       10.2.1.0 [200/0] via 172.16.0.2, 00:01:59
B       10.2.2.0 [200/0] via 172.16.0.2, 00:01:59
Provider_NY#sh ip route bgp
B    192.168.10.0/24 [20/0] via 192.168.2.2, 00:02:59
B    192.168.20.0/24 [20/0] via 192.168.2.2, 00:02:59
     10.0.0.0/24 is subnetted, 4 subnets
B       10.1.2.0 [200/0] via 172.16.0.1, 00:02:59
B       10.1.1.0 [200/0] via 172.16.0.1, 00:02:59

Suboptimal paths exist here too: customer’s routes are always reached via the local BGP peer.

Why does BGP prefer routes from the local peer? Let’s get more details about one of these routes:

Cust1_SF#sh ip bgp 10.1.1.0
BGP routing table entry for 10.1.1.0/24, version 2
Paths: (2 available, best #2, table Default-IP-Routing-Table)
  Advertised to update-groups:
     1
  65000
    192.168.3.2 from 192.168.3.2 (192.168.20.1)
      Origin IGP, metric 0, localpref 100, valid, internal
  65000
    192.168.1.1 from 192.168.1.1 (192.168.1.1)
      Origin IGP, metric 0, localpref 100, valid, external, best

Cust1_SF receives C2 subnet (10.1.1.0/24) from Provider_SF (eBGP) and from Cust1_NY (iBGP). Both routes have the same value for all attributes, the only difference is the way they have been learnt: internal (iBGP) and external (eBGP). The BGP decision process picks the external one: this is the rule.

Now it’s time to build an optimal routing table; we want to reach C2 and C3 subnets through SF provider’s router, C4 and C5 through NY router, and unknown destinations (1.1.1.0/24) via NY router too.

On the customer’s side we can achieve this objective by setting different LocalPref values on prefixes learnt at the AS border: SF router will set C2 and C3 with LocalPref = 300, other prefixes with LocalPref = 200; NY router will set C4 and C5 with LocalPref = 300, other prefixes with 250. We can do this using route-maps:

Cust1_SF:

ip prefix-list C2 seq 5 permit 10.1.1.0/24
ip prefix-list C3 seq 5 permit 10.1.2.0/24
!
route-map EBGP-With-ProviderSF_IN permit 10
 match ip address prefix-list C2 C3
 set local-preference 300
route-map EBGP-With-ProviderSF_IN permit 20
 set local-preference 200
!
router bgp 65001
 neighbor 192.168.1.1 route-map EBGP-With-ProviderSF_IN in

Cust1_NY:

ip prefix-list C4 seq 5 permit 10.2.1.0/24
ip prefix-list C5 seq 5 permit 10.2.2.0/24
!
route-map EBGP-With-ProviderNY_IN permit 10
 match ip address prefix-list C4 C5
 set local-preference 300
route-map EBGP-With-ProviderNY_IN permit 20
 set local-preference 250
!
router bgp 65001
 neighbor 192.168.2.1 route-map EBGP-With-ProviderNY_IN in

A clear ip bgp 65000 command forces BGP to update routes and, now, they are filtered by the route-map. This is the result:

Cust1_SF#show ip route bgp
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [200/0] via 192.168.3.2, 00:10:16
B    192.168.20.0/24 [200/0] via 192.168.3.2, 00:10:45
     10.0.0.0/24 is subnetted, 4 subnets
B       10.2.1.0 [200/0] via 192.168.3.2, 00:10:16
B       10.1.2.0 [20/0] via 192.168.1.1, 00:18:32
B       10.2.2.0 [200/0] via 192.168.3.2, 00:10:16
B       10.1.1.0 [20/0] via 192.168.1.1, 00:18:32
Cust1_NY#show ip route bgp
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [20/0] via 192.168.2.1, 00:10:32
B    192.168.10.0/24 [200/0] via 192.168.3.1, 00:11:01
     10.0.0.0/24 is subnetted, 4 subnets
B       10.2.1.0 [20/0] via 192.168.2.1, 00:10:32
B       10.1.2.0 [200/0] via 192.168.3.1, 00:11:01
B       10.2.2.0 [20/0] via 192.168.2.1, 00:10:32
B       10.1.1.0 [200/0] via 192.168.3.1, 00:11:01

We have wath we needed! But what’s up on the Provider’s side?

Provider_SF#sh ip bgp 192.168.10.0
BGP routing table entry for 192.168.10.0/24, version 21
Paths: (2 available, best #2, table Default-IP-Routing-Table)
  Advertised to update-groups:
     2
  65001
    172.16.0.2 from 172.16.0.2 (192.168.2.1)
      Origin IGP, metric 0, localpref 100, valid, internal
  65001
    192.168.1.2 from 192.168.1.2 (192.168.10.1)
      Origin IGP, metric 0, localpref 100, valid, external, best

Nothing! Each router still prefers local BGP peer to reach customer’s subnets. Our route-maps just set LocalPref on customer’s routers, so their influence is limited to just the way learnt routes are preferred each other. To let the provider use an optimal routing table we can set the MED attribute on outgoing BGP UPDATEs, so that locally connected subnets could have a better (lower) metric than others. As usual, we can do that using route-maps and prefix lists:

Cust1_SF:

ip prefix-list Net10 seq 5 permit 192.168.10.0/24
!
route-map EBGP-With-ProviderSF_OUT permit 10
 match ip address prefix-list Net10
 set metric 200
route-map EBGP-With-ProviderSF_OUT permit 20
 set metric 300
!
router bgp 65001
 neighbor 192.168.1.1 route-map EBGP-With-ProviderSF_OUT out

Cust1_NY:

ip prefix-list Net20 seq 5 permit 192.168.20.0/24
!
route-map EBGP-With-ProviderNY_OUT permit 10
 match ip address prefix-list Net20
 set metric 200
route-map EBGP-With-ProviderNY_OUT permit 20
 set metric 250
!
router bgp 65001
 neighbor 192.168.2.1 route-map EBGP-With-ProviderNY_OUT out

Then:

Cust1_SF#clear ip bgp 65000
Cust1_NY#clear ip bgp 65000

Wait until BGP does its job, then:

Provider_SF#sh ip route bgp | inc 192
B    192.168.10.0/24 [20/200] via 192.168.1.2, 00:06:05
B    192.168.20.0/24 [200/200] via 172.16.0.2, 00:06:04
Provider_NY#sh ip route bgp | inc 192
B    192.168.10.0/24 [200/200] via 172.16.0.1, 00:06:46
B    192.168.20.0/24 [20/200] via 192.168.2.2, 00:06:46

Now our provider reaches our subnets in the way we need.

Our subnets are still received from the local eBGP peer, but with different metrics:

Provider_SF#sh ip bgp | inc 192.168
BGP table version is 29, local router ID is 192.168.1.1
*> 192.168.10.0     192.168.1.2            200             0 65001 i
*>i192.168.20.0     172.16.0.2             200    100      0 65001 i
*                   192.168.1.2            300             0 65001 i

192.168.10.0 is the subnet locally connected to the provider’s router peer, so it has a low metric (200), but 192.168.20.0 is not, so it has a higher metric (300). Provider’s router prefers to reach that subnet via the internal link through the other iBGP peer, which receives the subnet with a lower metric (200) from it’s eBGP peer.

On the lab .ZIP file you can find two configuration sets: base and final. The first contains just a base configuration with IP addresses and iBGP on the provider’s side. The final configuration contains all commands reported in this post.

EDIT 2010-09-07: thanks to a friend, Akuavi Dossou, I found an error in the previous .ZIP file. Cust1_SF and Cust1_NY configurations in the “Base” directory were wrong, 192.168.3.0/30 IP addresses were assigned to the S0/1 interfaces instead of E1/1 and iBGP between the routers didn’t come up. Configurations in the “Final” directory were OK. I fixed the error and uploaded the new .ZIP file. Sorry for the unexpected troubleshooting exercise. 🙂

Download the Lab here.

The following two tabs change content below.
Italian, born in 1980, I started working in the IT area in the late '90s; I'm now a system and network administrator with a deep knowledge of the global Internet and its core architectures.

9 Comments

  1. James Huang says:

    Hi Pierky,

    It seems not quite realistic for the customer-SF (AS65001) to know the network topology of the other customers connected to provider-SF. Do you think the configurations can be improved by using community attribute as follows:
    (1) At provider-SF, set routes learned from its directedly customers with community attribte xxx and set routes learned from its iBGP peer with community attribute yyy.
    (2) At Customer-SF, set local-pref based on the community attribute for the route learned from the SF-provider.
    If community attribute == xxx, then local-pref == 200
    Else local-pref == 100

    (3) Do similar steps for provider-NY and customer-NY.

    — James Huang

    • pierky says:

      Hi James,

      yes of course your solution is the best one often used in real networks. This post was just an example from the book Internet Routing Architectures 2nd Edition.

  2. anuj says:

    This was really descriptive lab tutorial .
    Thanks for the post !!

    FREE INE CCNA LAB

  3. Fahad Alam says:

    thank you 😀

  4. Thanks to the post. I am studying to CCNP route and needed a example. This perfect! But i think that you have a wrong.

    Provider_SF#sh ip route bgp | inc 192
    B 192.168.10.0/24 [20/200] via 192.168.1.2, 00:06:05
    B 192.168.20.0/24 [200/200] via 172.16.0.2, 00:06:04 to reach network 192.168.20.0 throght to local ROUTE connected in interface 172.16.0.2.

    Thanks.

  5. CCNA Chennai says:

    Dear Pierky,

    Thanks a lot for your Cisco blog.

    I am preparing for my CCNP and now learning BGP. Could you please explain what exactly the meaning of “Multihoming”? The following text is copied from the Cisco CCNP ROUTE Quick reference book.

    ***************QUOTE***********************

    A site with a single ISP connection is single-homed. This is fine for a site that does not depend heavily on Internet or WAN connectivity. Either use static routes, or advertise the site routes to the ISP and receive a default route from the ISP.

    A dual-homed site has two connections to the same ISP, either from one router or two routers. One link might be primary and the other backup, or the site might load balance over both links. Either static or dynamic routing would work in this case.

    Multihoming means connecting to more than one ISP at the same time. It is done for redundancy and backup if one ISP fails, and for better performance if one ISP provides a better path to frequently used networks. This also gives you an ISP-independent solution. BGP is typically used with multihomed connections. You can take multihoming a step further and be dual-multihomed, with two connections to multiple ISPs. This gives the most redundancy. BGP is used with the ISPs and can be used internally also.

    ***************QUOTE***********************

    It says that Multihoming is connecting to more that one ISP at the same time.

    Thanks in advance…

Leave a Reply to Fahad Alam Cancel reply