DNS Blacklist filtering still rules, but for how long?

I came across a blog post from Virus Bulletin today, explaining that anti-spam solutions displayed significantly lower spam catch rates [in their latest test] than in other recent tests. To explain this drop, Virus Bulletin hypothesized that spammers are doing a better job at avoiding blacklists.

This reminded me of a blog post from John Levine, explaining why IPv6 will cause even more harm to DNS Blacklists (DNSBL). We also noticed a serious drop in the blacklist catch rate, reported a few weeks ago by our anti-spam team here at Vircom. And suddenly, I had an epiphany: relying on DNSBL filtering just can’t work! Or rather, it might be working now, but for how long?

Let me explain how it works. A DNSBL publisher collects spammers IPs by setting up some honeypots – some juicy email addresses – and waits for them to be spammed. When it happens, the senders IP is identified and the blacklist server starts screaming THIS IP IS A SPAMMER! and all anti-spam filters that listen to it start blocking it. This worked extremely well when a few IPs were sending huge amounts of spam.

Even a few years ago, when spammers began using botnets to send their messages from a large number of different IPs, DNSBLs adapted and maintained a high catch rate. According to Levine, the rate was about 80%, and up to 90% according to Vircom’s experts.

Nowadays, it feels like most of the anti-spam industry relies on DNSBL filtering and considers the importance of content filtering as secondary. And there are good reasons for this situation:

DNSBL filtering is good enough

Anti-spam companies have no interest in content filtering because it requires heavy R&D investment, while the DNSBL recipe is well known.

IT administrators have no interest in content filtering because it requires powerful, costly hardware while DNSBLs don’t.

End users have no interest in content filtering because it usually means a quarantine to manage, while DNSBLs don’t store the blocked messages.

So, everyone but spammers should be happy with the current situation, and we can only hope it will stay like this for as long as possible. But, I can’t help seeing some dark clouds coming, mainly because blacklist efficiency relies on two assumptions – which may prove to be wrong in the near future.

First, DNSBLs require a large volume of spam to be sent. This is logical: a spammer who targets 10,000 carefully selected email addresses has a much lower probability of hitting a DNSBL honeypot than one who targets 10 million email addresses. So, if the volume of spam drops, the efficiency of DNSBLs will most likely also drop. This is a serious issue given that Vircoms anti-spam experts reported that daily traffic on their honeypots dropped by 90% over the last few years, and that other major players like Cisco and M86 reported the same tendencies with respective drops of 60% over the last 18 months, and 50% over the last 10 months.

Second, DNSBLs rely on the same IP addresses being used to send multiple messages each. If a spammer were to send only one message from his IP, blacklisting it wouldn’t be so useful. But, as described by Levine, with IPv6 down the road and Internet Service Providers allocating ranges of IPv6 to their end users, an IPv6 botnet zombie may be able to use a different IP for each spam it sends. Spamhaus, the biggest RBL player, has already acknowledged this threat and explained that while today DNS-based blocklists are the work-horses of the spam filtering world, […] in the future under IPv6 Spamhaus sees DNS-based blocklists as part of a more sophisticated system of checks.

Eventually, the DNSBLs may become victims of their success. Their efficiency, combined with recent spectacular legal actions against major botnets (like the Rustock botnet last year), made the traditional buy Viagra spam less profitable and more risky. Today, if I were to join the “dark side”, I would most likely prefer a business model based on spear phishing: choose only a few thousand carefully selected targets, send highly personalized messages with a high probability of making it to users inboxes, and a higher chance of convincing my victims to do whatever I want them to do. DNSBLs have a harder time blocking this strategy.

And this feeling that there are fewer but increasingly dangerous scams around has been confirmed by a couple of recent experiences I had with phishing attempts.

Last February, a Registration Expiration message made it straight to my inbox. It was written in proper English, and contained personal info about me (my name, home address, phone number) while referring to a domain I own, asking me to pay $75 to renew my search engine optimization services. I had to carefully read the message before identifying it as a scam. Its sender simply collected the Whois domain information that I forgot to make private. It was simple, but really efficient. I wonder how many people fell into that trap?

A few months before, my father, who is renting a vacation house in the south of France, received quite a generic mail from a potential South-African tourist who was willing to rent his house for the whole summer. This was an unusual request, but my father agreed – provided the customer paid a part of the rent in advance. After a couple of back-and-forth emails, the customer explained that his bank wouldn’t authorize the money transfer unless my father paid the transaction fees first, which were as low as one third of the total rent! That’s when I got involved. The emails were sent using a Slovak email address. The senders physical address was in South Africa, while the phone number was from Ivory Coast. The bank was Ghanaian, but its address was in Abidjan, Ivory Coast again. It didn’t take long to understand that this was a phishing attempt, specifically targeting owners of vacation houses in France.

These spear phishing attempts, the most efficient ones I’ve seen in a while, outline the weaknesses of DNSBL filtering. I’m guessing these messages weren’t sent to millions of recipients and that none of the DNSBLs that protect my fathers inbox or mine ever saw them. These messages also outline the challenges that content filters face to differentiate scams from legitimate messages. Now, I wonder how long the anti-spam industry will consider DNSBL filtering to be the most efficient solution we can hope for, and that blocking 99.9% of spam is good enough, while the 0.1% that makes it through is far more dangerous than the usual annoying but harmless buy Viagra stuff.

Or maybe my experience is not representative, and you don’t share the same impression about the future of email filtering? I would be really curious to read your comments about your own experiences!