Malware

Knowledgeable humans are still the best spam filters

Spammers and phishers spoof the addresses, subject lines, and contents of legitimate emails from popular services. A simple rule of thumb can help spot the fakes.

Spammers and phishers spoof the addresses, subject lines, and contents of legitimate emails from popular services. A simple rule of thumb can help spot the fakes.


Both spam filters and increasing awareness of the most simple forms of trickery are allowing many users to more easily detect and avoid opening spam and phishing emails. While automated spam filters are far from perfect -- one must effectively choose between a filter that doesn't catch everything it should or a filter that catches things it shouldn't -- they are getting better at filtering out the simple stuff.

The spammers and phishers are not taking this lying down, however. As with the Cold War between the United States and the Soviet Union, there is an arms race going on between mass unsolicited emailers and the people trying to detect, and defend against, their efforts. Every time a new solution is available that provides better detection and protection, it turns out the spammers and phishers have already developed an even better way to get around your defenses and have already been using it for a while. Unfortunately, the defenders are always a couple of steps behind in the technological arms race. It takes human judgment to really beat the spammers and phishers.

The most knowledgeable and careful among us tend to be fairly immune to these problems. For instance, by reading all email in plain text, with zero markup parsing and no image displaying, some of us have rendered ourselves effectively untouchable by the efforts of phishers: it tends to be safe to open emails from such miscreants as plain text, since the scripting and markup tricks phishers use to get computers to do their dirty work for them without the permission of the user are rendered inert and ineffective when the email client does not parse the content as markup. The day may come when someone finds a way around even that level of caution, but by then the most knowledgeable about email security will probably have come up with an even better way to deal with the issue.

The fact remains, though, that no matter how well the most knowledgeable (and careful) among us can defend ourselves against phishers and spammers, the majority of email users have neither the understanding nor the tools to similarly defend themselves. More depressingly, many people who should know better refuse to use such tactics as viewing emails only in plain text at least until the safety of the email in question has been determined to be safe and genuine with certainty.

One of the keys to protecting yourself is to avoid acting impulsively when it comes to email. Another is to minimize the tendency to let the computer do your thinking for you, especially considering how bad computers are at approximating "thinking". Relying solely on safety rules imposed by an application developer is a recipe for failure, in part because by the time the newest version of an application gets to the end user the chances are good that these rules are already out of date. It is even worse when using email Web client software such as one of Microsoft's flagship offerings, Windows Live Hotmail. Such services typically "help" you by defaulting to opening new messages when users try to select a message or even when they simply delete another message in the same "mailbox". From a security perspective, this kind of email interface behavior is simply unacceptable, but it is essentially the norm for Webmail.

Going back to acting impulsively, however, a constant problem for phishing and spam email is the tactic of spoofing, or taking on the appearance of, legitimate emails from popular Web services. It is a constant problem, rather than a solved problem, largely because the efforts to spoof legitimate emails are becoming more sophisticated. This is an attack not only on the ability of spam filters to weed out the bad apples, but also on the ability of end users to recognize something "phishy" about the malignant emails -- at least until it is too late, and most users have already opened the email.

By curbing the tendency to act immediately and impulsively when dealing with such emails, a simple rule of thumb can help defend against the attempts to trick us into trusting an email we should not, in fact, trust. The rule is simple: have patience.

Specifically, whenever you get an unexpected email from something like Amazon, PayPal, Facebook, eBay, or even Chase Bank, wait a while before opening it. Wait at least 12 hours, in fact. The theory is quite simple: when botnets start sending out phishing and spam emails targeting users of specific services or customers of specific businesses, they tend to send everyone several such emails in a short period of time. If you get one such email now, chances are good that you will get a couple more in the next twelve hours or so, with the same subject line, or at least as subject line that is recognizably similar. Even if a legitimate email of the type being spoofed involves some kind of request for confirmation, such requests tend to take at least 24 hours to expire (usually longer), meaning that waiting somewhere in the range of twelve to sixteen hours gives you plenty of leeway. Even if you let requests in legitimate emails expire, the default tends to be to let things remain how you set them up in the first place, so that there is probably no harm done by missing the deadline.

This simple technique of having patience, and looking for duplicate phishing attempts, is especially effective if you have multiple email addresses. In that case, more than one of your email accounts may well receive such spam or phishing emails spoofing legitimate communications from popular Web services, making the clumsy deception even more obvious.

Keep in mind that this is not a foolproof method of detection. There may be cases where only one such email arrives in your inbox in the first twelve hours, leaving you still uncertain, so that opening the email to read it in all its HTML- and JavaScript-parsed glory can still be dangerous. As always, you should engage your brain before dealing with any unexpected emails, and exercise your best judgment. Waiting before even considering opening such emails, however, may well help you improve your effectiveness and accuracy in identifying spam and phishing emails that try to sneak in past the BS filter in your brain.

About

Chad Perrin is an IT consultant, developer, and freelance professional writer. He holds both Microsoft and CompTIA certifications and is a graduate of two IT industry trade schools.

26 comments
AV .
AV .

Knowledgeable humans are the best spam filter. None of the spam filters on the market are 100%. Reading mail in plain text does take a lot of the risk away, but most people won't do that. If you right click on the email before opening it, you can view the source code and see if the clickable links really do belong to the supposed email sender. Most times, its pretty obvious that they do not. I never click on a link in an email I'm not sure of, especially banking emails in HTML format. They look authentic, but it isn't worth the gamble. Its much better to open your browser and access the site that way. AV

richard.s
richard.s

How fitting that the TechRepublic email containing the link to this article was classed as "spam"! Since VirginMedia ISP switched to using Google-powered email, I've had the daily chore of rescuing genuine emails from the Google-powered spam folders. After yesterday's total failure of the POP3/SMTP email service, this morning the webmail spam folders contained several emails from the CNet/ZDNet/TechRepublic group.

sboverie
sboverie

A sign that the message is spam is how well the title is spelled or if it is blank. A lot of spam that I get has leetspeek spelling or really bad spelling to get around most spam checks.

david
david

Another indicator is when the real from address is not the organisation it is meant to be. I received one from service@paypaI.com . The real address is clearly not Paypal! Another variation is if it is from xyz@citibank.imaspammer.com. Anyone can add a well known name to the start of the domain name. The same applies to the real link address as against the displayed address.

thadb
thadb

We've been going about this the wrong way. The problem is that no matter what technologies or laws we come up with, there is too much money to be made in the mass mailing industry to ever combat it the way we've been going about it. We need to make it very difficult to make a profit by sending out spam. How about issuing licenses to hack/crack and destroy any isp/server that knowingly and repeatedly gets caught sending spam. How about levying heavy fines on companies who hire spammers.

howiem
howiem

A better rule of thumb is just to never click on any link in any email from any entity where you can or do conduct financial transactions. If you are not sure and think that the email might be genuine, don't be so lazy. Visit the site (preferably in a sandbox) using the genuine URL and locate the information discussed in the email. If you can't find it, delete the email, because since banks do not send you email about account problems, you will only miss out on a promotion at worst.

roch
roch

I check out all incoming messages on my BlackBerry before "receiving" them to my PC inbox. Most spam can be figured out from the subject line, but even if I have to open the message to see what its "selling", it is done without image downloads and with limited script execution. Not that I haven't been caught a few times, especially years ago when phishing first flooded out, but I am still the best spam filter for my inbox. (whether I'm knowledgeable or foolish and wasting a lot of time with my BB is a different conversation - and YOU are not my wife!)

JackOfAllTech
JackOfAllTech

If you have more than one account on the same domain, you can compare incoming emails. If you get an email on both addresses with the same subject and sender then - no matter how legitimate it may look - it's spam.

Sterling chip Camden
Sterling chip Camden

As a domain owner, I get all mail sent to my domain to any non-existent users. Most spammers do a dictionary blast of every possible name @camdensoftware.com. That's a case where a specific spam filter rule helps me out, because if it isn't to a valid address I know it's spam. It also shows me what messages are likely to be spam even if they're to my valid address, and it trains my Bayesian spam filter, too.

apotheon
apotheon

If you right click on the email before opening it, you can view the source code and see if the clickable links really do belong to the supposed email sender. Which email client is that? I know that's true in some of them -- but not all of them. It sounds like you're expecting everybody to use the same client. Don't just check links. Check images and scripts in emails, too. Does the email contain images that are hosted somewhere out there on the Internet? If so, opening an email can confirm that the address to which the email was sent is a good place to send more spam and phishing emails. If it contains scripts, and you don't check to make sure they aren't malicious, whole new worlds of hurt can be visited upon you. In my experience, 98% of HTML emails that the recipient doesn't expect receiving are spam or phishing emails. Meanwhile, most email clients used by people sending emails for legitimate purposes actually include plain text versions of the messages sent, in addition to any HTML versions that might be sent, so that people using text-only clients can read them. If you think people won't view emails in plain text, I think they're even less likely to view source before opening the emails -- if they're even using a client that allows them to do so.

SirWizard
SirWizard

When the grammar of the title is in Engrish, it's nearly always spam. The five messages each titled, "ALL MENS NEED THIS" or "PROTECT YOU'RE BANK ACCOUNT" get deleted without me wondering if I'm missing something valuable.

TobiF
TobiF

A lot of spam is sent from botnets. So owners of ADSL internet lines may be sending spam, without knowing. Not so nice to come and destroy their PC at home just because their child tried to download a funny little game a month ago...

Sterling chip Camden
Sterling chip Camden

If the message is plain text, the link is plainly visible. Check the domain name, and especially watch out for extra words in it before the TLD (e.g., bankofamerica.gynyf.com). If it's an HTML email, view the source to find the link. The text shown for the link may not match the href.

NexS
NexS

Something like Email> fake@domain.com = spam I like it.

AV .
AV .

If you add a registry entry in Outlook, it will allow you to view the entire source by just right clicking the email and choosing options without opening the email. No, I guess I didn't cover every email client, but this article does cover many of them. http://email.about.com/od/emailsourceaccess/View_the_Source_of_a_Junk_Email_in_Your_Email_Program_or_Service.htm My Comcast webmail allows me to see the entire source by right clicking on it. You never have to open the email. Not all HTML emails are malicious spam. Many are legitimate newsletters or advertisements from valid companies. They may be unsolicited, but they're not malicious in any way. Just annoying. You may sign up for 1 newsletter and they will share your email address with their affiliates if you neglect to tell them not to. You can unsubscribe from them if you don't want any more emails and they will remove you. This is only true for reputable companies. Real spam and phishing emails are malicious. They either want to infect your PC or rip you off. Things like banking emails made to look like the real thing, Nigerian email scams (which are plain text) and unsolicited emails with attachments. The rest of the stuff is just harmless junk mail. I have a lot of experience with spam. Part of my job, unfortunately, is to manage the spam trap. I see thousands of spam messages every week, most of it advertisements. All of my users view emails in plain text, but its not their choice. If it was, they wouldn't do it. They probably wouldn't view the source either, until they get burned. Then the light goes on. AV

Sterling chip Camden
Sterling chip Camden

My Bayesian spam filter says that the word "<html>" in an email gives it a spam probability of 98.3%. Try to be a little more precise. Seriously, is that a startlingly accurate informed guess, or do you have data on which you based this statement? Of course, I think the number would be lower for users who regularly receive valid HTML email from friends and family -- but maybe not much lower.

richard.s
richard.s

In my region, VirginMedia acknowledged the POP3/SMTP fault and its effects; although the webmail interface was still mostly working. - Perhaps your region is separate? - Perhaps you are not using the new SSL access? This service is still dodgy today; sometimes working; sometimes working only for some of my accounts. But, the fault(s) also seem to have upset the spam filters: This morning, most of the CNet/ZDnet/TechRepublic emails were classed as spam.

AndrewFisher
AndrewFisher

So does SirWizard's use of "without me wondering" where he should have written "without my wondering" indicate that his message is spam?

apotheon
apotheon

1. Make sure the plain text you're viewing is ASCII, and not Unicode. An ASCII URL can be spoofed using Unicode characters. 2. Depending on what a malicious email is intended to do, it may be too late to safely deal with an HTML email at all if you've already opened it in a manner that allows the HTML to render. The only way to be sure you're safe is to view it only as plain text.

Jaqui
Jaqui

if not valid_address then spam=yes if using spamassasin. and that is a built in filter option with it. :D or a require valid_address filter, where if the email isn't to a valid address it gets tagged as spam.

apotheon
apotheon

Actually, years and years of heuristic observational guesswork has taught me that, more often than not, in cases of tremendous majorities and of great cynicism, 98% is usually about right. edit: I've been calling it "The 98% Rule" for about a decade now.

santeewelding
santeewelding

Was perfectly acceptable to me. Simply another way of saying the same thing, only more actively, as opposed to your ossified "should".

apotheon
apotheon

1. I prefer nvi over Vim, these days. It's a smaller vi clone with a friendlier license. 2. Depending on your settings, Vim may render Unicode/UTF8/whatever cleanly anyway, so you may still run into character spoofing issues -- especially if you're using a "user friendly" OS like Ubuntu that tries to make everything as convenient as it can possibly be (including getting owned by phishing emails).

Sterling chip Camden
Sterling chip Camden

Most of the time, it's better to just go to the site's home page by keying in the URL in your browser, but some organizations send special links that you have to follow (usually some sort of confirmation link). Even if you copy/paste the link into your browser instead of following it, you can't be sure the link is OK unless you visually parse it first. I wish all such organizations would at least send a plain text version in addition to the HTML version, but most of these establishments see the web only as a way to provide more convenience, so they want to reduce the experience to a single click.

Editor's Picks