E-mail spam has quickly become a major headache for anyone
with an e-mail address. The total cost to the business world in terms of time
and effort spent fighting the spread of mass junk e-mail distribution is
astronomical—easily in the tens of billions. For the IT professional on the front
line of this battle, there are numerous filters and prevention methods at their
disposal. However, the spammers are ingenious and continue to evolve and refine
their mass distribution techniques in an effort to bypass whatever protections
may be deployed.

This continuing deploy/circumvent cycle has lead to what can
easily be described as an arms race between e-mail administrators, ISPs, and
other IT professionals versus the spam marketers and their shady tactics. With
this escalating battle as a backdrop, authors Bryan Costales and Marcia Flynt
wrote sendmail
Milters: A Guide to Fighting Spam, to help IT professionals get the
upper hand in their prevention efforts. A downloadable PDF of Chapter 3,
Setting Up a Bait Machine, from that book is available
from the TechRepublic Download Center.

In the following interview, the authors discuss the spam
problem and some to the prevention techniques they have found to be most
effective.

Interview

[TechRepublic] For
the TechRepublic community members, who may not have hands-on experience with
Sendmail servers, please explain, in general, how sendmail Milters work.

[Costales and Flynt] A sendmail Milter
(Mail Filter) is a stand-alone program that runs alongside sendmail. Because a Milter is a multi-threaded application, one Milter may support many forked sendmail processes running
in parallel.

Milters and sendmail processes can be put together like
tinker-toys. You may mix and match them in almost any manner to construct a
complex mail filtering environment from simple components.

A Milter generally screens all or
part of any mail message received and advises sendmail about each part. A Milter can “accept” a message or
“reject” it, or “continue”—that is, neither accept nor
reject the message at the present time. A rejected message can be “tempfailed” (the sender told to try again later),
“bounced” (rejected outright), or “quarantined” (held in
the queue pending human review).

A Milter may also add and remove
recipients, add and remove headers, and replace all or part of the message
body.

Multiple Milters can run on a single server. Each is called
in turn for each category of each message. For example, all Milters review the
connection and HELO information before any single Milter
reviews any other part of a message. Milters are called in the order defined by
a sendmail configuration file. When sendmail is configured to use a Milter, it sends each bit of connection information to the Milter. Among the information sent can be connection,
recipient, sender, host, header and body information.

When multiple sendmail processes are used in different roles
(for example, a queue processor and a listening server daemon), each can share
the same pool of Milters, but each might call them in a different order or use
different Milters from the pool.

Because Milters are so versatile, they are necessarily, and
unfortunately, somewhat complex. In our book, we try to make them very
understandable.

[TechRepublic] In
your book, you discuss spam and spam purveyors as moving targets because the
technology and techniques they use to bypass filters and other prevention
methods is continually refined and improved. Does this mean that, by its very
nature, spam proliferation can never really be completely eradicated, but
merely slowed to a manageable level?

[Costales and Flynt] Spammers are in it for the money and will go to
remarkable ends to achieve their goal. To illustrate, consider ISPs that white
list based on a list IP numbers. A spammer, when faced with such a site, will
remain well behaved for no matter how long it takes to be promoted to a
favorable place in that list. The wait can take weeks or even months, but the
difference between an unfavorable IP classification and a favorable one could
mean tens of thousands of dollars. Thus, the spammer is content to remain
inactive until a favorable classification is achieved, then the spammer will
awaken and send hundreds of millions of message. Sure, that address will be
shut down quickly, but not before the spammer has gained a great reward.

So long as it is possible to make money by abusing e-mail,
spammers (and today, phishers) will never be
discouraged. Just as laws against crime cannot eliminate crime, laws against
spam can never eliminate spam.

[TechRepublic] At
the beginning of your book you mention several other methods for fighting the
proliferation of spam besides sendmail Milters, including Domain Name Services
(DNS), Bayesian filters, screened URLs, and postage. Do you believe that the
best way to fight spam is use a combination of these methods? Is there a
combination that you believe works best? What new and improved methods are on
the horizon?

[Costales and Flynt] No single solution can eliminate all spam.
Clearly, given the wily nature of spammers, a broad based approach is always
best. Try simple spam fighting methods first to pick the low hanging fruit.
Some spamming sites, for example, send all spam from a fixed IP address. Block
that address (using sendmail’s/etc/mail/access
database) and a significant portion of your daily spam can be quickly
eliminated. Follow this kind of IP screening with Bayesian and URL screening
and (with daily maintenance) 90% of spam can routinely be eliminated.

The trouble is, the better you get at removing spam, the
more likely it becomes that you will wrongly reject valid email. False
positives are the bane of spam filtering. To rectify this problem, consider
saving rejected email for periodic human review.

Two current approaches to filtering spam use sender
identification and site validation. The theory used by both methods is that
spammers need to hide to be effective. Sender identification seeks to make the
source of spam easier to find. Domain Keys and similar schemes seek to
digitally sign email so that its source can be conveniently tracked. Neither
will suppress spam directly, but may embarrass some spammers out of existence.

The right solution for your site depends
how much spam and phishing affects your site and your
site’s customers. What does the problem cost and how much are you willing to
pay to reduce it?

[TechRepublic] Because
sendmail Milters is open source, what advantages does it have over proprietary
solutions, especially proprietary server-side filters? What disadvantages?

[Costales and Flynt]
Because they are open-source, Milters are more cost effective than
proprietary solutions. Many commercial proprietary solutions charge by the
mailbox (some over $1 per mailbox) and thus can become very expensive. Although
there is an initial investment to write Milters from scratch, maintenance of
Milters typically is less costly than proprietary solutions. Additionally, as
Milters become more popular for fighting spam, more Milters will be available
for clients to download for free.

Although
Milters were created to fight spam, they can also be used to solve many
problems:

  • A Milter
    makes it possible to archive a copy of all sent and received email
    (perhaps for security conformance).
  • A Milter
    can split out attachments and store them centrally, thereby reducing
    duplication and lowering storage costs.
  • A Milter
    can defer acceptance of non-company email until after business hours.
  • A Milter
    can gather and report email statistics for use in relational databases.

Over
the next few months, we expect many more Milters to become available from sites
such as www.milter.org and
spambook.bcx.com. As
spammers become more sophisticated, the open source community will insure
solutions will become readily available. Because this community is very large,
getting Milter support can also be both timely and
cost effective. These are just a few of the many reasons we feel open source
Milters are more advantageous than proprietary solutions for all clients
regardless of their size.

Chapter download

For a more in-depth look at what you can do to help your
organization prevent the spread of spam, download the book excerpt from sendmail Milters: A Guide for Fighting Spam–Chapter
3, Setting
Up a Bait Machine.