How to use BGP to achieve Internet redundancy

Configuring a redundant link to the Internet has given my company improved service and has reduced outages and related costs. This strategy also offers network administrators peace of mind as a bonus. Here is how to use Border Gateway Protocol (BGP) to produce similar results for your company.

With the proliferation of VPNs, e-commerce, and a multitude of other crucial Internet applications, access to the Internet has become mission critical for many organizations, and Internet connection redundancy is vital to ensuring the availability of these applications. Assisting in the justification for Internet connection redundancy is the decreasing cost of corporate Internet access.

BGP is one of the key tools for achieving Internet connection redundancy. When you connect your location to two different Internet service providers, it is called multihoming. When you multihome your network to two different ISPs, BGP runs on your Internet router(s) and provides redundancy and network optimization by selecting which ISP offers the best path to a resource.

Once you understand the basics of BGP and its terminology, the configuration required to multihome your network to the Internet is fairly straightforward. Let’s assume that you already have your primary Internet connection up and running. The general steps for implementing BPG multihoming are:

1. Obtain your ASN from ARIN.
2. Identify your network block of IP addresses. If you own these, then you have the right to advertise them on the Internet through BGP. If you are borrowing these from your provider, then you must ask your provider for permission before advertising them through another provider.
3. If you have a single provider, you are typically using a static route to connect to that provider. That provider is not sending you any BGP routes. Assuming that is true, you will have to request that your provider send you BGP routes. (Your provider will need to know your ASN and your remote router’s neighbor address. The neighbor is the IP address that your BGP process uses to communicate with.) Once you have the provider’s BGP routes in your routing table and you are advertising your network to your provider through BGP, you can remove your static route and have your provider remove their static route.
4. Next, assuming that you are multihoming on a single router, bring up your secondary provider. They can set it up so that they send you BGP routes. Again, they will need to know your ASN and your neighbor address.
5. Within the BGP table (database) on your router, you will see the routes from each of your providers. The best route in BGP is the route with the shortest AS path. (If the AS paths are identical, there is a tiebreaking procedure, but this is normally not the case.) The route that has the shortest AS path will be placed in your router’s routing table.

If you lose the connection to one of your Internet service providers, BGP’s keep alive packets will time out, and that BGP neighbor (from that ISP) will become down. Those routes will be removed from the BGP table, and thus, from your router’s routing table. Then, with only one set of routes in your BGP table, that provider’s routes (the secondary/redundant provider) are marked as “best” and placed in your routing table.

Normally, there will be some paths from one provider that are shorter than the other ISP, and vice versa. So your traffic will be distributed to the provider with the best AS path for whatever networks are advertised. However, if you are sending more traffic to a certain network (through one provider) than your link to that provider can handle, the extra traffic will not be distributed onto your second link. Using BGP metrics, you can attempt to do different forms of load distribution, but there is no real form of BGP load balancing.

Configuring BGP in the Cisco IOS
Now, assuming that you already have one provider, you have your ASN, you know what your block of IP addresses are, and you either own them or have permission to advertise them, here is a configuration example, using the specific Cisco IOS commands.

Let’s say that you have the following interface configuration on your Cisco router. (For explanation, note the comments and descriptions after the “!” symbol.)
interface Loopback0
description Loopback Adapter Used as BGP Neighbor Interface
ip address X.X.X.254 255.255.255.0
!
interface FastEthernet0/0
description Backup ISP Connection
ip address Z.Z.Z.1 255.255.255.252
 
interface Serial1/0
description Primary ISP Connection
ip address Y.Y.Y.1 255.255.255.252

Your BGP configuration might look like this.
router bgp 99999 ! Your ASNumber
no synchronization
no auto-summary
bgp router-id X.X.X.254
 
network X.X.X.0 mask 255.255.255.0 ! This is your block of IP Addresses
that you are advertising to the World
 
neighbor 456.456.456.456 remote-as 77777 ! Primary ISP AS number
neighbor 456.456.456.456 description Primary ISP Connection
neighbor 456.456.456.456 ebgp-multihop 2
neighbor 456.456.456.456 update-source Loopback0
 
neighbor 789.789.789.789 remote-as 88888 ! Backup ISP AS Number
neighbor 789.789.789.789 description Backup ISP Connection
neighbor 789.789.789.789 ebgp-multihop 2
neighbor 789.789.789.789 update-source Loopback0

To make sure that your BGP process is communicating with the correct BGP neighbor, over its appropriate link, you will need a couple of static routes. These static routes tell your BGP process to communicate with each of your BGP neighbor routers over their respective links.
ip route 456.456.456.456 255.255.255.255 Serial1/0
ip route 789.789.789.789 255.255.255.255 FastEthernet0/0

For explanations of what each command does, refer to Cisco’s documentation on Configuring BGP and the command reference for BGP.

Using BGP: Words of caution
Before looking at a sample Cisco IOS configuration of BGP, let me say a few words of caution about using BGP:

• BGP is the routing protocol of the Internet. Routing with full BGP routes on the Internet is a privilege. It means that you are an official member of the community of routers, routing all the important traffic of the Internet (yours and others). Being a member of that community, just like any other community, comes with responsibility. As a BGP neighbor, a couple of your responsibilities are:
  —To not let your route “flap” (go up and down) very often. As a full BGP router, whenever your network goes down and up, this is advertised to every other router on the Internet.
  —To only advertise the block of IP addresses that belong to you. Advertising someone else’s addresses could cause a loss of Internet service for that person’s entire network. You wouldn’t want someone else to do that to you, so you don’t want to do it to someone else.
• Everyone’s network is different; there are always some caveats or “gotchas” to be aware of. BGP and multihoming with BGP can be a complex network configuration. It is something to be duly investigated before proceeding haphazardly. (See the links below for more information.)
• The full BGP table of the Internet is huge. The BGP table is held in memory. If you are multihoming, then you will have two full copies of the Internet routing table. That means that your router must have enough memory to hold the entire routing table of the Internet—twice! In my experience, a router with 256 MB of RAM will suffice to do this. As an example, that is probably a 3600-series Cisco router with upgraded RAM. With full Internet BGP routing, you will have over 100,000 routes from each provider.

BGP resources online
Since BGP is the routing protocol of the Internet, there are some great resources on the Internet, including the following:

• TechRepublic: “Achieve network redundancy with BGP”
• Connect.com: Frequently Asked Questions on Multi-homing and BGP
• Joe’s BGP Page
• Cisco: BGP Technical Tips
• Cisco: Internetworking Technology Overview—Border Gateway Protocol (BGP)
• Cisco: Configuring BGP
• Cisco: Command reference for BGP
• Cisco: BGP Frequently Asked Questions (FAQ)
• Cisco: Sample Configuration for BGP with Two Different Service Providers (Multihoming)
• Cisco: Using Regular Expressions in BGP
• Cisco: BGP Case Studies
• Cisco: Internetworking Case Studies—Using BGP for Interdomain Routing