by Orlando Scott-Cowley
Spam volumes on the Internet are down on this time last year. Great news, we can all relax and stop worrying about our Junk or Quarantine folders or that missing million dollar order that might he hiding therein.
Brian Krebs wrote a great piece on the take down of the most prolific botnets, which is thought to be the main cause of drought in spam. It’s certainly true to say that since the likes of Spammit, Rustock, Coreflood, Pushdo and Bredolab have been knobbled the output of spam has been noticeably less.
Less spam is great news, but I’m worried. I suspect this eerie quiet in our spam and junk folders is a false sense of security, and one that is waiting to draw us into a more evil and harmful place.
Think about it this way. You’re a spammer…
Imagine you’ve been spamming people since 1997, persuading them to buy penny stocks, herbal enhancements and more recently fake AV products. You’ve been getting frustrated at the shrinking rate of return on your efforts, for the billions of spam messages you send you’re only seeing a 0.002% return or even less; mind you, at $30 for a bottle of those fake-little-blue-pills that’s still a few million dollars.
Why the decline? Well because we the vendors, are doing a better job of detecting and dealing with spam. Giving customers a 98% anti-spam SLA means we’re confident we can keep that junk and rubbish out of their inboxes. The same is true for personal or webmail accounts, providers are simply getting better at protecting users.
Then just when you thought things couldn’t get much worse someone shuts down your botnet, or the FBI takes away you hosting provider. Bad day at the office?
This is why I am worried…
Given the business challenges the spammers face today it’s no surprise we’re seeing a decline in the volume of spam. But are we? The figures we’re looking at here are related to spam volumes delivered over SMTP based email, and those have been on the wane for some time. The recent precipitous drop makes me feel uneasy about the spammers new business models. You might be surprised I’m using the word ‘business’ in relation to spammers – don’t be; this is their business, they have offices, employees, health-care plans, support lines and staff retreats just like everyone else.
These business models embrace all the latest social media trends. Spammers are simply jumping on the new mechanisms we’re using to communicate, social media gives them everything they need and in many cases an even more targeted audience who are trained to ‘like’ the same things their peers do.
The deeper impact of this switch to less well evolved communication channels, is that the classic AV and AS protections deployed at the corporate gateway are fast being made redundant. Their rules unenforced, their quarantines empty. The threats they protect against are getting onto the network via other means that in many cases are far less well protected. The point is that the spam isn’t going away, it’s just changing and adapting to the marketplace; the users might be breathing a sigh of relief when they look at their inboxes, but I can guarantee you they’re not doing the same elsewhere – Try tweeting the word mortgage or loan and see what happens.
The old money was SMTP email based spam, but just like everything else in corporate IT consumerization is taking over; spammers & scammers are simply keeping up with the trends.
by Nathaniel Borenstein
Standards work is generally conducted in what feels like slow-motion. More than a few highly-detailed conversations last for months or years. To those of us who’ve spent time in such conversations, it can be big news to learn that big news may be only a few months away. But for maximal, heart-stopping excitement, it should hint at the possibility of some day making real progress against spam.
That’s exactly what seems to be happening in the case of DKIM (Domain Keys Identified Mail), an emerging standard for cryptographically linking each message with the sending domain. In conjunction with some future developments, it could take a big bite out of “phishing” — unsolicited email pretending to come from a trusted institution.
Just a couple weeks ago — hot off the presses, in standards time — the chair of the IETF DKIM working group made the dramatic announcement (in the first paragraph) that things are going well. This means it could be as little as a few months before DKIM becomes a Draft Standard — a misleading term that describes the highest level that successful IETF standards generally attain. (MIME, for example, is a Draft Standard.) I think DKIM will be the first spam-focused standard to complete the standards process.
If you’re not accustomed to emptying the ocean with a cup, you can be forgiven if you’re breathing normally. But there are dozens of possible antispam measures not yet in use, and they will only work together effectively in the context of a very formal framework — a set of interlocking standards.
To oversimplify a bit: time favors the spammers because it takes far more computer power to examine a message than to send it. This advantage will probably last as long as Moore’s Law does. Eventually, inevitably, we will need to develop a more systematic approach integrating multiple interlocking technologies.
DKIM is, at long last, the first of those pieces. By itself, as its opponents are quick to tell us, DKIM will do NOTHING to stem the tide. But then, while a single rock can’t hold off a flood, a wall of them can.
So, it’s time to celebrate the near-completion of a decade’s work by some very good people. Even though it does almost nothing useful today. With all the energy I can muster, let’s hear it for DKIM: Hip-
[Full disclosure: Eight years ago I helped broker the peace treaty that merged DK and IIM into DKIM. And Barry Leiba is my friend.]
by Nathaniel Borenstein
I’m currently reading a fascinating book, Evolving God, by Barbara King. Professor King uses her years of experience studying apes as a starting point to explore how humanity evolved religion and ethics. It turns out that we share certain aspects of morality with apes, a sign that some of our basic morality evolved over eons, going back perhaps seventy million years.
It is because of this evolutionary history that our society doesn’t struggle to manage a “Right to Eat Babies” movement, because nearly all of us have inherited a nearly instinctual morality that characterizes baby-eaters as sick, evil, or both. Our moral battles instead focus on issues that have arisen relatively recently, in evolutionary terms. Abortion, for example, didn’t become a battleground issue until it became a safe medical procedure in the previous century.
Email technology is younger than I am, and I don’t seem to have evolved one bit. Our evolutionary heritage offers no guidance for many of the thorny ethical dilemmas email has created. Our inability to agree on the definitions of right and wrong surely complicates email immensely.
Take spam: everyone, save a few sociopaths, loathes it. But I’ll go way out on a limb here and reveal that I don’t consider spam immoral. It’s a bad idea that mucks up communication and creates incredible amounts of unnecessary work and expense. In many ways, it’s more of an question of judgement and etiquette than morality. If you leave a big box of candy with a child and he eats it all, he’s shown bad judgement and perhaps greediness, but I wouldn’t call it immorality.
Now, I’m not trying to start a defense of spam. I’m as happy as anyone to see spammers shut down, and the worst ones even jailed. But I see spam as being in large part the fault of a communication system that has eliminated all possibility of regulating behavior through pricing. Email is, in this sense, what the law calls an attractive nuisance. A technology deserves some blame for the antisocial uses it facilitates. Someone who is driving safely but over the speed limit deserves to get a ticket, but hasn’t acted immorally in my book.
This may seem like splitting hairs, but a difference of opinion over morality can easily grow into larger disagreements about laws and punishments. A thousand years ago, when abortion was a last resort because it usually killed the mother, discussions over its morality were largely academic, but they certainly aren’t today. I have heard — though I still can’t believe it — people advocate the death penalty for spammers. If that ever became a serious movement, the question of the morality of spam would take center stage for sure.
Because I believe that spam is caused by greedy, impolite people, I support filtering, voluntary authentication, moderate legal sanctions, and other countermeasures. Someone who believes spammers violate the laws of God would likely support harsher measures. Our evolutionary and cultural heritage gives us no guidance; there were no spammers in the savanna.
Each new technology gives us new ethical gray areas, further complicating our lives. Email has brought us several more ethical complexities, most more subtle than the morality of spam, which I’ll discuss here in the future. For now, though, I’ve got to go — there’s a chimpanzee who wants my help getting thousands of bananas out of Nigeria, and it seems like too good an opportunity to pass up.
by Nathaniel Borenstein
Recently, Facebook announced a 95% reduction in certain kinds of spam. Taken at face(book) value, that sounds like a tremendous breakthrough, but there’s less here than meets the eye, because the “certain kinds” are basically only those that are internal to Facebook, and the solutions are hard to generalize to the broader spam problem.
What Facebook has done is essentially allow users to provide feedback about which messages from Facebook applications are unwanted. By consolidating such feedback, Facebook can block further unwanted messages to most other users, and even sometimes completely block an antisocial application. If Facebook can be clever enough to learn like that, why can’t your email reader?
The answer is that it could, if only email weren’t so darned complicated. In the Facebook situation, all the offending messages are being both generated and read from within Facebook. The good folks at Facebook have complete control of the entire lifespan of such messages. They know exactly who sent the message, how many such messages were sent, and so on. None of this is true for your email reader.
The idea of letting users vote about spam is a good one, and not a new one; researchers at IBM and elsewhere have demonstrated the value of letting users vote about which messages are spam, and using those votes to decide which similar messages to block in the future. But those experiments have also highlighted the difficulties.
The world of email is one of many independent actors, interacting according to well-specified standard protocols, all of which are often ignored or misunderstood. If your mail reader gives you a button to click on when you think a message is spam, what should happen when you do so? Obviously your mail reader needs to send your vote (which may itself be wrong or accidental) to some server that collects it, consolidates it, and feeds the result into your spam filter.
But all of the actors in this scenario are heterogeneous. Your organization may have any number of mail reading interfaces, each of which needs to provide a button and behave similarly when it is pressed. You might be using any of a number of spam filters, which may or may not be prepared to accept voting data, for which there is no standard representation. Worst of all, the server that collects the spam votes can’t necessarily trust all the information it gets; your machine may be compromised by a virus, for example, that deliberately corrupts the antispam voting database by labeling good messages as spam or spam messages as good.
Facebook doesn’t have any of these problems when it deals with mail from Facebook applications to Facebook users. It can watch exactly what users do with messages, and map that back directly to the applications that send them. For similar reasons, spam wasn’t a big problem back in the day when email was often a closed garden, and AOL users could only send to other AOL users. A single authority in charge of everything makes it easier to enforce rules and policies. But who wants a single authority in charge of the whole Internet? The cure would be worse than the disease.
The lack of a central authority is one of the defining features of the Internet, and reflects its origins in the effort to build a network that could survive nuclear war. The result is a net that is remarkably decentralized, democratic, and chaotic. The only way to end the chaos would be to regiment the net to an unprecedented degree, essentially to guarantee strong authentication for everyone who sends an email or does anything else on the net. This would be nice for anyone who hates spam, but more importantly, a boon for any government that wants to crush dissent, or any corrupt organization that wants to halt all leaks and criticism. That’s a terrible tradeoff, but I’m not terribly worried about it ever happening. The net’s design favors the most powerful force in the universe: chaos. I wouldn’t bet against it.
CC Image via jurvetson on Flickr
by Nathaniel Borenstein
Recently I wrote about the big kerfluffle where SORBS put MessageLabs on a blacklist, and MessageLabs’ customers’ emails stopped going through to SORBS’ users. I suggested that customers shouldn’t be quite so quick to point fingers, because email is a complicated business and accidents can happen to anyone.
But there’s another lesson to be learned as well. Our customers — and I’m sure some of our savvier competitors’ customers — were scarcely affected. The recipients most badly affected, almost by definition, were the ones that were overly relying on a single source of information about what is and isn’t spam. Even the most well-run, well-intentioned service will occasionally make a mistake, but it’s less likely that two will make the same mistake simultaneously. If you only absolutely block mail when two independent sources say to block it, you’re much less likely to be causing the kind of critical situations we saw last week.
At first blush, this would seem to suggest that the promise of cloud providers and appliance vendors — to take these worries out of your hands — is a false one. But in reality that depends on the way those third party providers are conceptualizing their own roles. If they are themselves making sure that a wide variety of factors are considered, you’ll probably get better results than if you did it yourself.
It’s easy, simple, and probably a mistake to give a single blacklisting agency total veto power over mail entering your site. But it’s reasonable to expect that your service provider is in fact basing such decisions on multiple sources of information. Any antispam company can, if it so chooses, base its decisions on multiple factors; the only incentive in the other direction is the potential cost of that information. When costs leads to short-cuts, decisions may sometimes be made based on isolated bad information.
At Mimecast, we subscribe and give weight to several independent blacklists, but we don’t give any of them absolute veto power over mail to our customers. Among other things, we automatically whitelist the email addresses with which our customers have communicated in the past. This means that if an email sender’s site had been blacklisted, we would be more likely to block most mail coming from that site, but would still allow mail from known correspondents.
In the current state of the art, whitelisting past correspondents simply trumps broader blacklists. No anti-spam technique is perfect; issues of identity spoofing are always present, and future countermeasures by the spammers may make this technique less valuable some day. Fighting spam well means you’re running hard just to stay in place; last week’s events give me even more confidence that we’re running in the right direction.
As I’ve said many times, email is a very, very complicated business. That’s a good reason to outsource it, to be sure, but only to a provider with a healthy respect for the complexity of email today, and a commitment to evolve along with it into an even more complex future.