TechRepublic Tutorial: Understanding sendmail

Discover how sendmail works.


Sendmail has been around almost as long as UNIX itself. The authors of the famed O'Reilly "bat" book compare it in complexity to the "Gordian knot" of ancient history. It's true; sendmail is a fairly complex program. A casual read of /etc/sendmail.cf by mere mortals can send the uninitiated screaming away in fear. But like most things, if taken in small bites, with a methodical approach, we can get beyond that complexity and come to an understanding of what sendmail is all about. I'm not saying you'll be a master after one short Daily Drill Down, but you should have a better comfort level with the program, should you need to deal with it.

Note
In Greek legend, the Gordian knot was the name given to an intricate knot used by Gordius to secure his oxcart. Gordius, who was a poor peasant, arrived with his wife in a public square of Phrygia in an oxcart. An oracle had informed the populace that their future king would come riding in a wagon. Seeing Gordius, the people made him king. In gratitude, Gordius dedicated his oxcart to Zeus, tying it up with a peculiar knot. An oracle foretold that he who untied the knot would rule all of Asia. According to a later legend, Alexander the Great cut the knot with his sword. From that time, "cutting the Gordian knot" came to mean solving a difficult problem.

What is sendmail?
Sendmail is an MTA, or Mail Transfer Agent. Its sole purpose in life is to move mail from one system to another or deliver mail on a single system from one user to another. This sounds simple enough, until you consider the wide variety of systems and network protocols out there and how sendmail manages to communicate with most of them. Most UNIX-derived operating systems come with sendmail out of the box, configured for the most common configuration. In many cases, you can just use it and not worry about how it works. If you have a valid Internet domain name and sendmail is serving just one box properly configured with the fully qualified domain name, sendmail should just magically do its thing.

Where you can run into trouble is if you need to "tweak" sendmail's functionality or if you need to upgrade due to security issues and build from source.

If you think about it, e-mail has been quite a magical revolution in communication. Unlike with the phone, you can still communicate with someone even if you're not both available at the same time. Unlike traditional "snail mail," message delivery can happen in minutes; if you're vacationing in Australia, you can still read the e-mail sent to you in Milwaukee, provided you have access to the Internet.

The definitive site for all things sendmail is sendmail.org. This site has links covering where to get sourcecode and documentation, as well as links to other helpful resources. If you have a large organization and want a commercially supported product, the sendmail author has spun off a corporate version, Sendmail, Inc., offering additional features, including graphical configuration, support, and alliances with other key vendors to provide virus scanning and other extras.

The most current version as of this writing is 8.12.0.Beta7. I'm going to download and discuss the current stable version: 8.11.3.

Compiling sendmail
Compiling sendmail from source is pretty easy, provided you're set up with the necessary tools to build programs from source:
tar -xzf sendmail.8.11.3.tar.gz
cd sendmail-8.11.3
sh Build


If you need special library or include options, they can be included while calling the script with the -I or -L flags and specifying your options.

There are extensive notes on other options, as well as various compiler/OS quirks here.

Once sendmail is built, you need to configure it. Sendmail uses m4 to build its final /etc/sendmail.cf file from .mc source configuration files. This allows you to define some fairly complex mailer behavior by simply defining an m4 option and then building your /etc/sendmail.cf. The alternative is to hack at /etc/sendmail.cf manually, which is sometimes okay for a simple change, but unless you feel completely confident fooling with stuff like this, I’d recommend against doing so. (Of course, if you're a Perl hacker, you're probably wondering why they chose to make the file so readable.)

Some of the options at the top of the /etc/sendmail.cf file are not quite so cryptic and can be easily changed right there. The downside of even making minor changes in the configuration file is that, if you ever need to regenerate from the m4, your changes may be lost.

So, from the newly created sendmail-x.x.x directory, run the commands
cd cf/cf
cp generic-linux.mc mysite.mc


which will copy the generic-linux.mc file to the mysite.mc file. By doing this, you will ensure that you still have a copy of the original, should you need to start over.

You should pick a template file that most closely resembles your environment. I'm building on a Linux system, so generic-linux seems appropriate.

Let's take a look at the template file.

Not too much to be seen here. Remember, this is just a generic template. For many sites, this would be enough. Let's talk about the directives in the file:
  • ·        divert(-1): This just means to ignore anything following, up until a
    divert(0) command appears.
  • ·        VERSIONID: This drops the version information in the resulting configuration file.
  • ·        OSTYPE: This is very important. If you don't properly define the OSTYPE, you'll have a number of problems getting things up and running.
  • ·        DOMAIN: You can enter your domain name here or leave it as generic and let sendmail get the name from the system.
  • ·        MAILER(local): The local mailer is always included automatically.
  • ·        MAILER(smtp): It should be noted that, if you intend to add other FEATURE directives, the MAILER directives should be at the end.

That's it. Each one of these directives will generate from one to many lines in the resulting .cf file, which you build with this command.

You'll end up with about a 1200-line myfile.cf, similar to this one. You'll also notice in the above output the commands you would need to run if you wanted to manually process an .mc file into a .cf:
/usr/bin/m4 ../m4/cf.m4 mysite.mc > mysite.cf

Recent versions of sendmail keep the configuration files in /etc/mail, rather than just /etc. To finish your install, back up your current sendmail.cf and the sendmail binary. (Usually /usr/sbin/sendmail in Linux.) Now copy your new configuration file over and install the new binary, after stopping sendmail. (You'll need to either su to root or use sudo for these steps.)
/etc/rc.d/init.d/sendmail stop
cp mysite.cf /etc/mail/sendmail.cf
cd ../../
sh Build install


There are also a number of utilities in the directories under sendmail. These include makemap, mailstats, vacation, and others. Contrary to what the INSTALL document says, these will also be installed when you run the install script in the top-level directory.

Using database maps
If you are using any database maps, you'll need to rebuild them with makemap from the original text source, (You do keep your original text source, don't you?) to make sure the format is compatible with your new setup.

This is generally done like this (example is using the mailertable database—more on this later):
makemap hash /etc/mailertable.db < /etc/mailertable

hash is one of three database options that makemap can use. The others are dbm and btree. Sendmail expects the hash format by default, but it can read a number of database formats, provided you've compiled support for them into the sendmail binary and you specify the lookup technique in the .mc file configuration. You can even leave the lookup table in plain text format, but lookups become quite slow if your list is very large. The implied database lookup command is:
hash -o /etc/mail/somedatabase.db

But you can also define any number of other lookup techniques, including calling an external program:
dbm -o /etc/mail/virtusers
program /path/to/program -d data.file


The option being looked up is passed as the last parameter by sendmail.

Using procmail
Sendmail expects procmail as the local delivery agent for Linux. Chances are, if you were already running sendmail, you have procmail installed. You can always check quickly:
[stew@omnibook stew]$ which procmail
/usr/bin/procmail


If not, you can grab a precompiled RPM for your distribution or get source from the procmail site. Procmail is much more than a simple delivery agent with quite powerful filtering features, but we'll leave that to another discussion.

For more information on procmail, see Jack Wallen, Jr.’s, Daily Drill Downs, “All the wonders of procmail, part 1,” and “All the wonders of procmail, part 2: Lockfiles and nondelivering recipes.”

Host names and aliases
One item not mentioned in the sendmail documentation is the existence of /etc/mail/local-host-names. Sendmail will fail on startup without this file. It should contain the names of the hosts you will be accepting mail for. You can easily create one, like this.

You will also need an aliases table. Chances are, you already have one in /etc, so you just need to move it to /etc/mail. Otherwise, you may see the following error message when trying to send or check mail status:
hash map "Alias0": missing map file /etc/mail/aliases.db

After you copy your aliases file to /etc/mail, run newaliases. An aliases file is mandatory, with a minimum of the following entries:
# Basic system aliases — these MUST be present.
MAILER-DAEMON:  postmaster
postmaster:     root


Many times, you'll add an entry to have root's mail forwarded to an administrator's account:
# Person who should get root's mail
root:          stew


Often, you may want to create an alias for a group of people:
accounting:   greg, shirly, ashley

Newaliases is a symbolic link to /usr/sbin/sendmail. Running it will notify sendmail of your changes.

The format of the aliases file is the same format used for many of the database lookup tables for sendmail. Entries on the left are matched with those on the right. Sendmail either replaces instances of the left-hand side entry with the right-side data or treats such entries according to rules on the right-hand side.

That about does it for installation. If you've followed all the steps correctly, you should be able to start up your new sendmail binary:
[root@omnibook /root]# /etc/rc.d/init.d/sendmail start
Starting sendmail:                            [  OK  ]


If you take a look at /var/log/messages, you should see something like this.

Testing sendmail
You can try to telnet into port 25 and see the daemon at work (provided you haven't disabled telnet and replaced it with ssh).

And you can try sending yourself some mail:
[stew@omnibook stew]$ mail stew
Subject: test
test
.
Cc:


If things are set up correctly, you should find the test message in your favorite mail program's inbox.

If you should notice a long delay in starting sendmail or checking the queue status with mailq, there is probably something wrong with your host name setup, and sendmail is trying to find out the local host name.

Other options
That's it for the basics. Let's go back and look at some of the options available when starting up sendmail. Like most UNIX command-line programs, sendmail accepts a number of switches that affect its behavior:
  • ·        -d Debugging mode: The "d" is to be followed with various numbers to change the debugging level by category and level. With no options, the default is 0-99.1, which outputs debugging data for all categories at level 1.
  • ·        -v Verbose mode: Sendmail will output additional information when running.
  • ·        -q Process the mail queue: This is especially useful if you are on a dial-up and have sendmail configured to hold mail until explicitly told to process it. This can also include a code to tell sendmail to process the queue at a regular interval (q1h = hourly).
  • ·        -bd Run in daemon mode
  • ·        -bt Run in address testing mode
  • ·        -n Don't use the aliases file
  • ·        -Xfile Log all mail traffic in the specified file (useful for debugging)
  • ·        -Cfile Use an alternate configuration file

There are many more. The sendmail man page covers them in more detail.

Many systems will run sendmail from its startup script with the -bd and -q options only; it’ll look something like
/usr/sbin/sendmail -bd -q15m

which means run as a daemon and deliver mail every 15 minutes.

Since I mentioned the case of a system not connected to the Internet at all times, let’s quickly touch on how you would set this up. We would still like local mail to be delivered immediately, but we want sendmail to queue up all mail to addresses outside of our network until explicitly told to deliver it. We'll need to add some entries to our .mc file:
define(`confCON_EXPENSIVE', `True')
define(SMTP_MAILER_FLAGS, e)
define(`SMART_HOST', `smtp.myisp.com')


What we are asking for here is that the HOLD_EXPENSIVE flag in sendmail.cf be set to true and that all SMTP mailers be marked as expensive. We're also going to pass all the outgoing mail to the SMART_HOST server at our ISP—smtp.myisp.com.

The effect on the resulting sendmail.cf will look like this.

And finally, if you look at the definitions of the SMTP mailers near the end of the sendmail.cf file, you'll note the addition of the e flag for expensive:
Msmtp,             P=[IPC],              F=mDFMuXe

The local mailers will not have the e flag.

Now all outgoing mail will be held, until you run the command:
/usr/sbin/sendmail -q -v

By default, relaying through sendmail is now disabled. Relaying from/to any host used to be fairly commonplace, before the wild explosion of the Internet and the corresponding spam e-mail we all receive. Most mail servers now deny relaying by any host outside their own domain or certain "trusted" hosts. In the next installment, we'll cover how to fine-tune this behavior, as well as how to control usage of the mail system by folks both inside and outside of your system.

Conclusion
We've touched on a handful of features that you can enable in the .mc files, as well as a bit about the database files that sendmail uses. As I mentioned earlier in this Daily Drill Down, sendmail is an extremely flexible, configurable program with a long history. There are many more features and "hacks" that can be enabled. The O'Reilly "bat" book has 1,000 pages of sendmail and related documentation, which may give you an idea of just how deep the subject can be. Some people have written off sendmail as a dinosaur and tout the virtues of other mailers, such as qmail. These mailers are good too, but sendmail still runs many a high-volume mail site quite satisfactorily. In the next Daily Drill Down, we'll going into more features, including how to fine-tune your site configuration, how to enable masquerading, some security issues, and log analysis. In the meantime, you should try grabbing a copy, building from source, and browsing the sample .mc files.