The reason IPv6 is needed may cripple blacklisting. FUD or for real? Michael Kassner investigates the potential for harm that IPv6 could pose for fighting spam.
No, some of us have to actually work during the day, I replied, with more than a little sarcasm.
"Well, your highness; when you get a chance. Read it."
My friend knows which buttons to push. I had to find out what he was talking about. I opened my Register e-newsletter and right at the top: IPv6 intro creates spam-filtering nightmare.
More bad vibes
Wait a minute. That's only one source. After checking, other media outlets seem to agree: "New IPv6 protocol could complicate e-mail spam filtering" and "Will IPv6 render blacklisting obsolete?" Whoa, it is big. The fact that IPv6 has an available address space I can barely fathom may screw up blacklisting.
I had to get on this bandwagon. Jumping into action, I shot out emails to my special corps of experts asking them:
Will blacklisting become obsolete because of the obscenely-large address space created by IPv6?Anup Ghosh (Invincea): IPv6 does create a much larger address space from which spam and malicious sites can park themselves. However, current techniques for blacklisting known spamming/malicious sites will likely be unaffected. Blacklisting techniques are largely agnostic to the size of the address space.
An IP address is added to a blacklist when the address is verified as a bad site. The issue with blacklisting is by the time a domain makes it to a blacklist, its IP address has already changed.Giorgio Maone (NoScript): Blacklisting has always been the weakest form of protection in security, on principle. A much larger address space just makes this more evident. But, it's hardly news. For example, Mark Ranum, father of the firewall explains in this old editorial.
I believe statistics method to recognize spam, e.g. Bayesian filters, are the only really scalable solution, for now at least.Joe Klein (IPv6 Security Researcher): The IPv6 spam-list problem has been solved for some time. Many of the non-IPv6 aware blacklist companies currently block a single IP address in IPv4. In IPv6, each home user receives a single /64 bit address, where the first 64 is the user's unique network and the last bits are the user's local network. All the black list needs to do is block the network or the first 64 bits of the address.
I have been teaching this information in my IPv6 hacking class for over a year.Johannes Ullrich (SANS Internet Storm Center): Blacklisting will have to be thought over when it comes to IPv6. In IPv4, blacklists for the most part, list individual IP addresses. This will not work very well in IPv6.
In IPv6, the address is broken down into two parts: The first half is addressing the subnet. The second half is addressing the individual host (interface) on that subnet. A user could pick any address within that subnet and some operating systems will pick a random interface ID whenever they reboot to assist with privacy.
This "second half" is 64 bits long and allows for 4 Billion squared possible combination (the entire IPv4 internet only has 32 bits or 4 billion worth of addresses).
I think blacklisting will need to be able to block subnets. That way, it doesn't matter which IP address within the subnet the spammer uses. IPv6 luckily uses fixed subnet sizes (/64 being the smallest, and /42 typically assigned to organizations). This may lead to some collateral damage but it is probably the only way to make blacklists effective.
On the other hand: Blacklists haven't really been that terribly effective in the IPv4 world. Maybe we will finally think about more systematic spam fixes than blacklists.Cameron Schmauch (EdgeWave): In my opinion blacklists and whitelists have been obsolete for several years. This is not so much because it's hard playing whack-a-mole with the spammers, but rather that the lists usually aren't maintained in such a way as to aggressively prevent False Positives (FP).
EdgeWave (Powered by Red Condor) hasn't relied on third-party blacklists for anything other than supplemental information. We do employ methods for sussing out IP blocks that are operated by spammers. Blocking on IP can be very effective and efficient, but it should not be the mainstay technology if keeping FPs to a minimum is your top priority.
In our records (which go back over half a decade), about 22% of our categorizations were due to matching IP rules (not necessarily origin only). Over the past year, that figure has plummeted to only 6%. The reason this statistic has dropped so dramatically is spammers are already good at usurping the resources and reputation of others to get their dirty work done.
You can't outright block mail from abused addresses if legitimate mail also uses those paths. You have to rely on more sophisticated techniques because a single dimension just doesn't provide enough information to make an absolute decision on message disposition in many cases.Kassner: Since spam and malware filtering is a big part of EdgeWave's business, I decided to ask Cameron a few more questions.
Do we need to worry about it?
Short answer: I'm not worried about it. We have largely abandoned such lists. Much of the worry seems to be simply about the size of the new address space. However, from what I've seen reported on this issue, many tacitly assume that spammers will somehow be able to pop in and out of this address space at random.
Yet, practical limitations inherent in doing this could prove more difficult than is actually warranted for the spammers to get their messages delivered. Spammers, like all other things, follow the path of least resistance.
Moreover, this is an old trick. Various spammers already tread very lightly with respect to their address space holdings, sending lower volumes and using what I like to call "IP crop rotation" to avoid using the same addresses too often which would otherwise make them easy targets for blacklist maintainers.
The real Achilles's heel of spam is in its volume in combination with a specific intent over small time domains, not simply where it comes from.
The other thing is that IPv6 adoption will likely take many years. I don't see any problem with mail administrators "blocking first and asking questions later" with regard to inbound IPv6 connections. This could be in the form of aggressively gray listing IPv6 connections from unknown sources, or outright blocking until reputation has been established.
That's sort of what I imagine will come out of best-practices from the early adopters. In terms of our solution, we've believed that the best way to reliably filter unwanted mail without collateral damage is with multiple layers of defenses, behavior analysis, multi-scale feedback systems; and of course, human-in-the-loop real-time analysis. Anyone who is still using IP blacklists as a primary filtering solution is likely already having a bad time.
Do you see any other issues with IPv6?
I think its adoption will exacerbate short-comings in the DNS system, routing and miscellaneous security systems for example. But those problems are somewhat mundane.
The more interesting impact, I think, will be from the fact that everything will start to come online. The vacuum of such a large address space with the advent of embedded computing and cheap bandwidth is the far more interesting aspect of this whole transition.Final thoughts
Well, this piece ended up in a different place than I thought. It seems IPv6 blacklisting will work in a fashion. More to the point, it's obsolete. My expert resources came through again. I can't thank them enough for their willingness to set the record straight.
Now, you will have to excuse me. I need to make sure someone reads this.