Networking

Black hole routes: The good, the bad, and the ugly

While you may not have a network as large or as critical as YouTube to manage, unexpected network downtime or lost network traffic is never a good thing. That's why you should understand what a black hole route is, how it can help you -- and how it could hurt you.

The entire YouTube network recently went down, across the globe, for about two hours -- thanks to a mistake made by an ISP in Pakistan. That mistake involved a black hole route accidentally distributed around the world.

While you may not have a network as large or as critical as YouTube to manage, unexpected network downtime or lost network traffic is never a good thing. That's why you should understand what a black hole route is, how it can help you -- and how it could hurt you.

What happened with YouTube?

If you haven't already heard what happened with the YouTube incident, it's a very interesting story from an IP routing and troubleshooting perspective. Because plenty of articles are available that discuss the incident, I won't go into detail. But if you need a quick review, check out these resources:

What is a black hole route?

In the 1979 movie The Black Hole, what went into the black hole didn't come out. It was a scary thing that all the characters wanted to avoid. In the world of IP routing, a black hole is also a term with a negative connotation -- trust me, no one wants traffic going there either.

In its simplest form, a black hole exists on a network when a router directs network traffic to a destination that just "throws away" the traffic. The classic interface used on a Cisco router to do this is the null0 interface.

In mathematics, null means to have zero numbers in a set. In computers and networking, however, null doesn't mean zero; it means no value or nothing.

A Cisco IOS router also has an interface called null0. When traffic goes to that interface, the router just discards it. Thus, the null interface on the Cisco router is the "black hole."

How can a black hole route help you?

Obviously, you can direct traffic that you want to get rid of to a black hole. In fact, this is what happened with the Pakistani ISP and YouTube's data. While there are many ways to discard traffic, I suspect what happened was that the IPS sent all YouTube traffic to null0 and then accidentally shared the route with other ISPs throughout the Internet using Border Gateway Protocol (BGP); all YouTube traffic ended up discarded, all around the world.

Black hole routes can help you by dropping malicious traffic if you're under attack, such as in the case of a DDoS attack or a worm attack. While you may be able to do this just as well with an access control list (ACL) since routing works in the forwarding path of the Cisco router, you can use the black hole route to drop the same traffic while incurring less of a performance impact on the router. (Because the ACL processing is higher in the Cisco IOS order of operations, the ACL would serve the same purpose but take more router resources to do it.)

Take advantage of a black hole route with the Cisco IOS

We typically configure black hole routes in conjunction with BGP; BGP is the routing protocol of the Internet, and most of the malicious traffic is on the Internet. However, anyone can configure a black hole route with just a single statement.

Here's the simplest form of a black hole route:

Router(config)# ip route 1.1.1.1 255.255.255.0 null0

This statement sends all traffic arriving on this router to the null0 interface -- in effect, discarding it and sending it to the black hole. Let's look at an example of using this in a simple network.

Let's say your Cisco IOS router connects you to the Internet, and your network users are using an online P2P file-sharing service that you don't want them to use. Rather than creating an ACL, using content filtering, or an application-based firewall, you could simply drop all traffic to that domain by looking up the IP addresses used and entering a route to null0.

You could also redistribute this route into your dynamic routing protocol and have it sent to all other routers on your network. Then, all routers would send traffic to your router, and your router would drop that traffic.

Keep in mind that if you're using this approach, you don't want to alert malicious users that you're blocking their traffic. You could configure the following on the null0 interface:

Router(config)# int null0
Router(config-if)# no ip unreachables

For more information on using black hole routes, check out these Cisco PDF resources:

How can a black hole route hurt you?

Of course, black holes as bad if you're on the other end of this discussion and you're the one sending the traffic. Of course, if you're sending malicious traffic, your network deserves to go to a black hole. On the other hand, if it's a mistake, as was the case with YouTube, having traffic sent to a black hole could cost you and your company significantly.

Conclusion

With BGP, there's no foolproof way to stop an accident or malicious attack from taking down the traffic to an Internet destination. However, more BGP security may eventually come in the form of Secure BGP.

In the meantime, it's important to understand what black hole routes are, how they work, how they can help you, and how they can hurt you. Whatever you do, you don't want to fall into a black hole!

Want to learn more about router and switch management? Automatically sign up for our free Cisco Routers and Switches newsletter, delivered each Friday!

13 comments
jcbel
jcbel

Good article - I read the linked NetworkWorld article and highlights the lack of process to verify that incoming BGP data is valid. Sounds to me like all sources are trusted implicitly?

ihdaniels
ihdaniels

Very concise and enlightening!

dawgit
dawgit

Especially for Ciscoites. I know Cisco is a widely used system, but there are two assumptions here; The Whole World uses only Cisco, and the Pakistani incident involved Cisco. Do we really know that was the case with the YouTube affair? -or- Is this an excuse YouTube is using for something that they should have prevented. An excellent article though. I enjoyed it. -d

catseverywhere
catseverywhere

Good info, with special thanks to cc, excellent links, good find.

tom.marsh
tom.marsh

What does "Valid" mean? If I configure my router to accept routes from a remote AS (and this is required to use the protocol) you are relying on that organization to configure their router correctly. And hopefully, most ISPs at the backbone tier have a good enough BGP admin in-house to get their routers configured correctly so you're not vulnerable to mistakes by others like this... In other words: The only way to exploit this is if you're already a trusted ISP somewhere, or have the capacity to become one before such time as you'd want to distribute your wrong route, and even then you would need to identify, in advance, which upstream provider would accept routes for networks you don't control. A very very messy (and trackable) attack, unlikely based on how easy it is to rent a botnet and launch a DDoS attack anymore...

Dumphrey
Dumphrey

and I have only one problem: how do you block P2P programs with a single IP address sent to the black hole, or even 20 Ips? distributed peer systems are the norm now, not server client models. This technique would stop the old Napster, but have no effect on modern p2p such as bit torrent.

CrimsonPaw
CrimsonPaw

I gotta agree, this is good info to know. For those like me who only have a year or two working in the world of Cisco it's good to hear of these little nuances HERE rather than having to research a mistake I made.

S,David
S,David

This is theory, not practical knowledge here, so don't flame me too bad if I get it wrong. My understanding is: The ISP(s) that supply Pakistan's internet connection should never have allowed a "downstream" router to change a route it did not control. An AS number lookup would have shown that the router broadcasting the change did not control the route, and the change should have been dropped. But, they are not the Lone Ranger here. The only problem I see with BGP is that it will work "properly", i.e. do what you want, without the configuration being 100 percent correct and bulletproof.

Joe Chiarani
Joe Chiarani

According to http://en.wikipedia.org/wiki/Border_Gateway_Protocol BGP is a standard. Most ISP routers (Cisco, 3Com, Juniper) support this protocol. The configuration may different. This article is just an example to configure Cisco router. In youtube case other provider's router or Pakistani can be any that supports BGP.

Neon Samurai
Neon Samurai

Admittedly, some shmuck downloading video and his latest music cravings on the work network is not acceptable but is there a valid use for such software on your network? I know torrents are very popular for Linux and BSD based OS distributions and other large and legal files. I think blocking may come down to monitoring rather than a nice clean firewall rule. In the case of torrents, you can watch for the seeding announcements from your internal staff. The bandwidth spike should be a good indication also.

tom.marsh
tom.marsh

Let me preface all of this by saying it was after a long career in network administration for smaller organizations whose WANs were simple star-topologies using static routing... But my first time using BGP was on a large MPLS-WAN and the provider didn't mention to this BGP neophyte that the configuration they advised me to use assumed that the remote sites got their internet access from a circuit at each location rather than across the WAN, as we did. So their config didn't advertise a default route. In reviewing my configs, the routers were from the existing WAN so they already had default routes entered statically, which I dutifully changed to the appropriate next hop for the new MPLS-WAN and considered myself pleased. Oops! First day, we go live, everything works except--the Internet! After troubleshooting, we realized the error... MY static routes were pointing to the right next hop, BUT that router didn't have a route to 0.0.0.0 that I was allowed to use, so the traffic...just...died. We had to do a little quick switch and advertise a route to 0.0.0.0 across the WAN. ...and now it works swimmingly! But yeah, BGP is a beast. I Shudder To Think how much fun it is to use BGP on a public, not-access-controlled network like the Internet. I'm getting more comfortable with it nowadays, but you are quite correct that there is a learning curve. I'm sure errors like the one in this story are quite common.

kdaugharty5
kdaugharty5

Yes this was good info, For I only have been in Cisco's World(LOL) for about 6 months now, and they don't teach that in any book I have seen.

Editor's Picks