Web Development

DNS: Patched but not totally fixed

DNS isn't out of the woods yet; there's a proof-of-concept exploit that successfully poisons the cache of a fully-patched and up-to-date DNS server. In order to keep everyone up to speed, I'd like to describe the attack vector and some possible fixes that we will be hearing about.

In an earlier article, "DNS: The Internet Dodged a Bullet, Thankfully," I described what many are calling a very serious design flaw in the DNS protocol. Quite simply, the bug allows attackers to poison a name server's DNS cache in a few minutes. After which, attackers can redirect unsuspecting Internet users to malicious Web sites that are set up to harvest personal information or facilitate the downloading of malware.

Fortunately, DNS developers rapidly came up with a fix and patched all major versions of DNS, including Cisco, BIND, and Microsoft. The patch increased the complexity of spoofing a DNS query response, by randomizing both the Transaction ID and the query port, making it 65,535 times more difficult to poison the DNS cache.

New attack vector

Well, it appears that's not enough. A Russian physicist, Dr. Evgeniy Polyakov, was able to spoof a fully patched BIND name server into accepting an incorrect IP address in a query response. Dr. Polyakov published the details of the attack on the blog "Successfully Poisoned the Latest BIND with Fully Randomized Ports!" The attack involves the following:

"BIND used fully randomized source port range, i.e. around 64000 ports. Two attacking servers, connected to the attacked one via GigE link, were used, each one attacked 1-2 ports with full ID range. Usually attacking server is able to send about 40-50 thousands fake replies before remote server returns the correct one, so if port was matched probability of the successful poisoning is more than 60%.

Attack took about half of the day, i.e. a bit less than 10 hours.

So, if you have a GigE LAN, any trojaned machine can poison your DNS during one night... "

Ten hours is a huge improvement when considering it takes less than a minute to poison an unpatched name server. Some security experts are of the mind-set that the patch still introduces enough entropy. Others are saying that Dr. Polyakov's test was unrealistic. Two servers hammering a recursive name server over a GigE link isn't a typical Internet connection and certainly not readily available to most attackers.

Dr. Polyakov has a convincing reply. To get the required bandwidth, computers on the same network as the DNS servers could be compromised using trojans. Those computers would then transmit the 40-50 thousand query responses required to carry out the attack, doing so during an evening or weekend to help evade detection. Another possibility is the use of botnets whose consolidated bandwidth would be sufficient to carry out the attack.

I submit that even though Dr. Polyakov's proof-of-concept test may not be realistic, the very fact that it's possible to subvert a recursive name server at all is very disconcerting. As I mentioned more than a few times, if there's even the slightest chance that DNS query responses are corrupt, it introduces all sorts of doubt in whether a displayed Web page is the actual one. I sense that most of the DNS experts feel the same way, because they are working on new fixes that should strengthen DNS to a point where cache poisoning is virtually impossible.

Possible fixes that will not work

If you read the comments on Dr. Polyakov's blog, you would see that there's an active dialogue going on promoting many possible solutions, several of which are interesting as they attest to the overall condition of DNS, especially the repercussions precipitated by possible changes to the protocol. I'd like to start out with the fixes that will not work as they allude to the fragility of DNS:

Debouncing is a process where the querying name server sends the authoritative name server the exact same query twice. It's a strange term, but it's a good approach, allowing the querying name server to check the validity of the query responses. If both responses match, it's a good bet that the responses are valid. If the responses don't match, then it probably means there's a cache-poisoning attack taking place. At that point, other more involved processes could be used to determine which response to accept.

Using TCP/IP and its three-way handshake will easily eliminate cache poisoning. The handshake authenticates both points of the connection, unlike UDP (used by DNS) which will accept any query response as long as it has the correct IP address and port number in the returning packet.

Great, either approach sounds like a good solution and shouldn't be that hard to implement. Except there's a catch, Dan Kaminsky and other DNS researchers feel that

"Debouncing is similar to the 'run all DNS traffic over TCP' approach -- seems good, up until the moment you realize you'd kill DNS dead. There's a certain amount of spare capacity in the DNS system -- but it is finite, and it is stressed. Absolutely there's not enough to handle a 100% increase in traffic over the course of a month."

In explanation, the existing worldwide DNS network (employing UDP and requiring only two packets per DNS query and response) is currently using more than 50% of available bandwidth. The debounce approach (double the number of packets) would immediately saturate the network, bringing the Internet to a grinding halt. If the debounce approach has that affect, there's no way using TCP/IP would work. It requires up to seven packets per DNS query and response. Besides not working, these solutions highlight how the existing DNS system may have other upcoming issues if the number of users continues to grow at the present rate.

Possible fixes that should work

DNS Security Extensions (DNSSEC) are a series of IETF specifications that can be added to the original DNS protocol specifically to increase security. DNSSEC uses Public Key Infrastructure, which allows digital signing of returning query responses from authoritative name servers. The querying name server can then check the digital signature against information it has and determine if the responding name server is valid.

DNSSEC has several other security enhancements to help protect DNS from other threats as well. You may be surprised to learn that DNSSEC has been around for almost 10 years, thus asking why hasn't it been implemented? Well, that's where it gets a bit sticky. DNSSEC is very complex and has implementation issues, requiring significant work to deploy it worldwide. Most DNS experts feel that DNSSEC or a revised version of it will eventually be used. DNSSEC just can't be deployed and tested soon enough to lessen the very real threat from cache poisoning.

0x20 proposal

I learned of the 0x20 proposal (very intriguing idea) from Dan Kaminsky and Steve Gibson. They both mentioned how the 0x20 proposal would add significant entropy to the DNS query and response. If you remember, the whole point of the recent changes to DNS was to increase the payload and port randomness of a DNS query and response. Doing so makes it more difficult for an attacker to flood the querying name server with enough fake response packets before the authoritative name server's response arrives.

The 0x20 proposal makes use of the Internet draft "Use of Bit 0x20 in DNS Labels to Improve Transaction Identity." Dan Kaminsky explains how this is possible:

"Basically, this idea from (I think) David Dagon and Paul Vixie notices that DNS is case insensitive (lower case is equal to upper case) but case preserving (a response contains the same caps as a request). So, we can get extra bits of entropy based on asking for wWw.DOXpaRA.cOM, and ignoring replies containing www.DOXPARA.com, WWW.doxpara.COM, etc."

This means there would be three components of a DNS query that introduce randomness: the Transaction ID (16-bits), the port number (16-bits), and now the queried Fully Qualified Domain Name (FQDN). Using the FQDN for added entropy is unique, because the total amount of randomness is completely dependent on the length of the name. Thus, longer domain names will have more entropy. For example, a FQDN using www and .com will add at least 6-bits of entropy, three from www and three from .com.

What makes the 0x20 proposal especially appealing is that it doesn't require any changes to the authoritative name servers. Authoritative name servers already preserve the case-sensitivity of FQDNs, by returning FQDNs to the querying server with each letter in the same case it was received. Only the DNS applications used by recursive name servers (already patched once) will have to be updated. All that's required is to add the ability to check if the letters in the returned FQDN are in the same case as the FQDN sent. Besides being a simple fix, the 0x20 proposal doesn't take up any additional bandwidth, and that's a significant plus.

Final thoughts

As I continue researching DNS and the underlying processes, the fragility of the DNS protocol becomes ever more apparent, which is alarming if you consider how utterly reliant the Internet is on DNS working properly. Ultimately, DNS experts seem resolved to have an improved variation of DNSSEC securing the DNS protocol. I suspect the 0x20 proposal will finds its way into the DNS protocol long before that, especially with the publicizing of Dr. Polyakov's successful cache poisoning of fully patched BIND name servers.

-------------------------------------------------------------------------------------------------------------------

Michael Kassner has been involved with wireless communications for 40 plus years, starting with amateur radio (K0PBX) and now as a network field engineer for Orange Business Services and an independent wireless consultant with MKassner Net. Current certifications include Cisco ESTQ Field Engineer, CWNA, and CWSP.

About

Information is my field...Writing is my passion...Coupling the two is my mission.

40 comments
Michael Kassner
Michael Kassner

I can't verify this but, there's evidence being discussed that some ISPs are capturing user DNS queries and resending them to the ISP's designated name servers. I was listening to a Steve Gibson "Security Now" podcast and this was the portion that really caught my attention: "Question from a listener: Long ago I changed my static DNS entries in my Linksys home wireless to the IP addresses of OpenDNS. So during the recent discussions of DNS vulnerability I thought I had little to worry about. However, the other day I fat-fingered a web request and got a Charter Communications "not found" result page. Wait a minute. Confused by this, I began to research. My router settings had not changed, and my client machine's IP config showed DNS set to the router IP and the two IP addresses of OpenDNS. Yet when I went to the DoxPara test page I found my DNS server was vulnerable to DNS cache poisoning, with all the requests being sent out over a single port. Anyway, Whois found that the IP address identified by DoxPara is owned by Charter Communications. LEO: Well, that's - huh? That's interesting. I guess that's who's hosting his site. My questions are many. How could Charter intercept these? Oh. Maybe they don't own his site. STEVE: Exactly. LEO: How could I stop them from intercepting my DNS requests and use a security-aware DNS service? What are my options? So let me recap to make sure I understand. He's using OpenDNS, so theoretically all his DNS should go through it. But for some... STEVE: Well, let's say he's configured to use OpenDNS. LEO: For some reason Charter is intercepting these. What's going on? STEVE: This is a problem. This means that, okay, and think about it, DNS is over UDP, that is not connection oriented and that has no TCP connection. And his ISP could do anything it wants with his packets. So they are receiving DNS queries coming from his network, out of his router, with a destination IP on the packet of OpenDNS. They know that it's a DNS query because it's aimed at port 53, which is a universal DNS query port. All DNS servers are listening for incoming UDP and TCP traffic on their port 53. Charter, for whatever reason, perhaps for the purpose of intercepting and putting up their own advertising page, they're ignoring the destination IP of his DNS packets and changing them, rewriting them to their own. So that... LEO: Oh, man. STEVE: ...no matter what he does, no matter what DNS configuration he uses at home, Charter is ignoring the destination IP, rewriting his packets on the fly so that they go to their DNS server. And if he, as he says, "fat-fingered," which I think means typo... LEO: Mistyped it, yeah. STEVE: Mistyped it. Then instead of his browser being told there's nothing at that IP, he's given an IP, his browser is given an IP of a server that Charter is running that brings up their own interception page for whatever purposes they choose. LEO: Usually advertising. VeriSign did this for a while, and it was very... STEVE: And, boy, they didn't do it for long because it really upset people. LEO: A lot of ISPs do that. And they say, well, we do it as a convenience, we have a much better page. And it's true, if you don't do that, then you get that stupid Internet Explorer page saying I can't get anywhere. STEVE: Okay, well, two things. First of all, you said many ISPs do this? LEO: Yeah. STEVE: The only way to do it is rewriting DNS. LEO: Ah, interesting. Of course, yeah, of course. STEVE: So that means all the ISPs that are able to present their own page are doing so by preventing you from going to the DNS server you have asked for. And secondly you'll notice that he ran the test at DoxPara, and it said all your requests are coming from one port. So not only is his ISP screwing up his deliberately configured-for-security DNS, but they're aiming it at an insecure server. LEO: They're doing a bad job. STEVE: Exactly. They've got a spoofable server, and they're forcing all of their customers to use their misconfigured spoofable server." I thought this might be of interest to the members as some have mentioned that they were getting strange returns on the DoxPara DNS randomness test. If this is true than certain ISPs are hijacking DNS queries to known good DNS servers and forcing users to obtain query responses from insecure DNS servers that aren't even randomizing the source port.

Dumphrey
Dumphrey

it seems good, but I am unclear on where its implemented, in the browser by the client, or on an isp level dns server? Where does the case randomization get initiated? Time for google i guess...

Jaqui
Jaqui

as a quick fix to allow site visitors to verify the site would be to post the site's ip address, and suggest that visitors ping the site name and verify they are getting the ip address posted. it doesn't stop the poisoning, nor is it a perfect solution, but if the site puts it's ip address up in an anti-bot "captcha" block it can help to improve visitor confidence until the dns system is fixed.

JCitizen
JCitizen

I assume OpenDNS is not a quick fix.

aaron.newton
aaron.newton

Has there ever actually been a confirmed case of this? Quote: BIND used fully randomized source port range, i.e. around 64000 ports. Two attacking servers, connected to the attacked one via GigE link, were used, each one attacked 1-2 ports with full ID range. Don't DNS servers have firewalls? It seems like this would be easily fixed by blocking a particular IP range that send too many incorrect requests very quickly. Or have I missed the point completely?

techrepublic
techrepublic

In the article, you say: "At a minimum, the FQDN will add 6-bits of entropy, three from www and three from the top-level domain characters (.com, .net, and so on)." This assumes that every queried FQDN starts with "www" and ends with a three-letter TLD, which is not an accurate assumption. TLDs can be as short as 2 letters (eg. most country code TLDS, .us, .uk, .ru, .de, etc.) and the second-level domain component can be as short as a single character, but IIRC, it must be a letter if it only has one character. Also, many query FQDNs do not contain the "www" prefix, nor do they need it in many cases, even for web traffic. Thus, a domain name like g.cn (Google in China) represents an example of the minimum bytes of entropy added by the 0x20 proposal, at 3 bytes. Regards, Jon Heese

bboyd
bboyd

gigE unrealistic? How about distributed computing via storm worm... 10hours or maybe just a few seconds.

seanferd
seanferd

I wonder how long until patch. Or is this something adjustable in configuration? Socially, I'll be watching to see how this plays out in the security community, media, and various fora. How critical will this vulnerability be rated? What reactions will we get to the rapid disclosure, rather than the notify-and-wait or high-secrecy project models of disclosure? What about Kris Kaspersky's vulnerabilities? Will Zone Alarm firewalls freak out again? It'll be an interesting ride.

Michael Kassner
Michael Kassner

I like the 0x20 proposal a great deal as it's a clean hack and gains a great deal of entropy for virtually no cost. It's a fix though and not a solution. DNS is just too important to have stop-gap measures all of the time. The fix would be applied to recursive name servers or any device that queries authoritative names servers

Michael Kassner
Michael Kassner

I like your line of thought, but wouldn't the user have to do that after actually being at the web site? So if the web site is spoofed, so could the the IP address that's supposed to be checked. Does that make sense?

Michael Kassner
Michael Kassner

As far as I know the latest attack is so new, (you heard it here almost first) that all DNS applications maybe vulnerable. I haven't read anything further, but I understand that all versions are going through the testing process. Also there's no way for any of us to test our IPS's DNS server against this attack as of yet. If any of the members have heard otherwise I really appreciate hearing about it.

Michael Kassner
Michael Kassner

The attacker spoofs the IP address of the authoritative name server using a malformed packet (via raw sockets). Remember the DNS query response needs to have the correct IP address and correct port or the querying recursive name server will not accept the response. So, you can't block that port or IP address.

Dave the IT Dude
Dave the IT Dude

This is a brute force attack (possibly also a form of DoS).After a set threshold (3 attempts from the same IP address?) that IP address gets blocked. If the attack takes 10 hours on such a fast network - with a direct connection that means a LARGE number of packets need to be sent - even changing your IP address in a pure class A network will soon soon fall foul of the limit. Make it so the count can only be manually reset by an admin who is notified of failled attempts - this may help in locating bots. Having an auto-reset after a time period would allow attackers to just space out their attempts (if it is a big target, does the difference between 10 hours and 10 weeks really matter?) Based on the above article, it looks like this researcher has only proven the success of the previous fix. He has basically used the same attack vector and shown that, even on the latest and greatest equipment it still takes several hours (on average) to be successful.

Michael Kassner
Michael Kassner

I forgot two simple words "for example" Thanks for pointing it out, Jon.

Michael Kassner
Michael Kassner

It's sort of a catch-22 syndrome. DNS is rapidly reaching bandwidth capacity, so logically you increase that. Which in turn creates more capacity for an attacker to use in trying to beat the authoritative query response back to the recursive name server.

rball
rball

I'd be interested to see how the mainstream media would cover this (if they ever did) - I always shudder when they try to talk tech but something like this, I can only imagine how they'd mangle the explanation. Then everybody I know would be asking me "Is it true the Internet is poisonous?" or "is it safe to go on the interweb anymore?" or some variation.

Michael Kassner
Michael Kassner

I thought using caps randomization on the FQDN was totally cool. Someone was indeed thinking outside the box. All sorts of extra entropy at virtually no cost. In fact, randomizing the FQDN that way really helps people that have not been using query port randomization because their NAT routers and firewalls wouldn't allow it or the NAT router de-randomized the query port. As for Kaspersky's claim, it appears that he's pulling a Kaminsky and will not reveal the jewels until the "Hack in the Box" security conference in October. Maybe it's the same sort of thing and he's giving vendors time to fix DNS. His claim is that both Transaction ID and query port randomization aren't enough. The DNS experts already knew adding query port randomization is just a stop-gap fix. As is randomizing the FQDN. They couldn't say that we have to do this as it raises the security bar, but it's still broken. I must admit though Kaminsky in a sense did allude to that on his web site. DNSSEC or a variation will be the ultimate answer, but that will take a huge mount of work and that's after they figure out several irritating deliver issues.

Michael Kassner
Michael Kassner

He Sean, What did you think about Charter and spoofing the user DNS?

JCitizen
JCitizen

with McAfee. Impractical of course. I'm just trying to make light in my moment of terror! :(

Jaqui
Jaqui

it's not perfect. most people don't visit new sites that often, they hit a number of sites regularly, these sites they use daily they would see the ip in the catpcha box and might then twig if the dns cache serving the site is suddenly poisoned by an ip change in the captcha box. Like the regular visitors to TR, most would notice an ip change, if the spoof site used the correct ip for TR hen a ping would show it as a spoof. Big sites, like Amazon, Ebay, Paypal, Google, Yahoo, MSN would improve visitor confidence with it as a temporary measure to help protect visitors, the little guys wouldn't be getting the traffic levels to make spoofing them worthwhile generally.

seanferd
seanferd

I found it before I found your post here. Again, an excellent piece of work.

Michael Kassner
Michael Kassner

I have another very interesting piece coming up this week hopefully. It's another one of those that everyone should be aware of.

pgit
pgit

Big media will never report this, it's just not sexy enough. So what, the entire planet could suddenly be cast asunder, in the dark, unable to communicate or conduct commerce... ho-hum. Are there any interns involved?

seanferd
seanferd

Simple idea, using an existing option. Very efficient thinking. I hope the additional entropy will buy enough time to really implement a new model for DNS, be it some version of DNSSEC or something new (or currently unknown).

Michael Kassner
Michael Kassner

My next article is about SSL certificates and should be out today some time.

JCitizen
JCitizen

That's a great story in and of itself!

Michael Kassner
Michael Kassner

It's certainly an interesting subject and I wish I understood it more completely.

Michael Kassner
Michael Kassner

That's a good idea, just as long as they don't use DNS queries to the authoritative name servers. (Sorry, my poor attempt at humor.)

seanferd
seanferd

They are repeatedly and massively requesting pages from the affected sites, which will prevent legitimate requests from succeeding. Any number of methods could accomplish this. They could be using botnets or dedicated machines to repeatedly connect to the sites. I suppose you could actually ask for millions of people to repeatedly try to access the site and get the same results, but automation is much simpler and more reliable if one is serious about the DDoS attack.

Michael Kassner
Michael Kassner

I suspect that DDoS and just bombing DNS servers would be more what's going on. There's no real need to be subtle about things, unless there was an underlying reason like covert ops I suppose.

Michael Kassner
Michael Kassner

If so, yes it was a DNS cache poisoning attack. The idea was to totally overwhelm the DNS server under attack with fake query responses in order to see if he could get a collision. Which means that one of the fake query responses had the correct Transaction ID and query port. That packet will get accepted along with the payload which carries the IP address of the attacker's DNS server. From that point on, when a query is sent for that specific domain, the attacker's DNS server is now authoritative and most likely will return an IP address for a spoofed site. It boils down to a race between the attacker and the real authoritative DNS server. With the attacker having to guess the Transaction ID and query port.

bboyd
bboyd

Hopefully the Georgian internet has been hardened. Did DNS server poisoning serve as an attack vector?

Michael Kassner
Michael Kassner

I suspect that it will be yet another race. With the proof-of-concept attack validated, it only requires methods that will make it more efficient.

Editor's Picks