SolutionBase: Securing your organization's SMTP gateway

If you're using an internal e-mail server, you might be leaving it open to abuse without knowing it. Here's what you need to know in order to secure the SMTP gateway on your e-mail server.

In a world in which network administrators go to extreme measures to secure computer networks, it's important to acknowledge the idea that some security measures are more important than others. For example, given a choice between fixing an obscure unchecked buffer vulnerability, and applying a patch that renders the latest virus ineffective, I would have to choose the virus prevention patch. I'm not saying that you should ignore the hypothetical unchecked buffer vulnerability that I just talked about, but you have to set priorities. In doing so, you must look at things like what the odds are of a particular vulnerability being exploited, and what the consequences are if the vulnerability were exploited. With those two considerations in mind, my opinion is that your network's SMTP gateway is one of the areas that should receive the most attention.

Who cares about the SMTP gateway?

So what's so special about the SMTP gateway? First, it's a portal into your private network from the outside world. Second, SMTP gateways are under almost constant attack. If you don't believe me, then think about this; where do most of the viruses come from? That's right, they come in through E-mail. What about Trojans? Well, Trojans can be delivered through malicious Web sites, but they are also commonly delivered through SPAM messages.

Viruses and malicious SPAM messages are only the beginning as far as ways that SMTP gateways are attacked though. Another common exploit against SMTP gateways is message relaying. Message relaying refers to a technique by which spammers passes mail through your mail server in an effort to obscure their own identity. Relay attacks can cause a couple of different problems for you. First, if a spammer is passing SPAM through your server, the bulk mail is going to consume a tremendous amount of resources. This includes things like Internet bandwidth, SPU time, and disk space.

The other problem with mail relay attacks is that SPAM messages that pass through your server appear to come from your network. If you allow this to happen, then your mail servers have a very good chance of being blacklisted. This means that anyone who subscribes to an anti SPAM blacklist (this includes most companies) will not accept messages (including legitimate messages) from your mail servers.

Still another threat to your SMTP gateway is a denial of service attack. This occurs when someone floods your mail server with so much junk that the server lacks the capacity to handle legitimate messages.

Other threats to your SMTP gateway come from within your own network. For example, you might have an employee who decides to start sending SPAM as a way of making a little extra cash on the side. Likewise, you may have employees unintentionally spreading viruses via SMTP mail. After all, most E-mail viruses automatically mail copies of themselves to all of the E-mail addresses stored on an infected machine's hard drive.

These are just a few of the countless threats against your organization's SMTP gateway. Hopefully, you are beginning to understand why I said that the SMTP gateway is one of the areas of your network that disserves the most attention when it comes to security.

Author's Note

Now that you have an idea of the threats against SMTP gateways, I want to talk about some ways that you can secure your gateway against those threats. Before I get started, I want to point out that there is no way that I can possibly cover the topic in its entirety in an article like this one. I would need a fairly good sized book to do the topic any real justice.

The other thing that I want to mention before I get started is that the available security mechanisms differ depending on what type of SMTP gateway you have in place. For the purposes of this article, I will assume that you are using a Microsoft Exchange 2003 Server that is running a Windows Server 2003 operating system. I am also assuming that your Exchange Server is running third party anti SPAM and anti virus protection.

What does Exchange do by default?

OK, so let's start out by checking out some of the SMTP security features that are available directly through Exchange Server. To access these security features, open the Exchange System manager and navigate through the console tree to your organization | Administrative Groups | your administrative group | Servers | your server | Protocols | SMTP | Default SMTP Virtual Server. Now, right click on the Default SMTP Virtual Server (unless you are using a different virtual server for your SMTP traffic) and select the Properties command from the resulting shortcut menu. When you do, you will see the Default SMTP Virtual Server Properties sheet.

The General tab

By default, the properties sheet's General tab will be selected, as shown in Figure A. At first glance, the General tab doesn't really look like it has any special capabilities by default. The tab does give you the option of logging SMTP traffic, but you must weigh the benefits of having such a log against the impact that logging SMTP traffic will have on your server. Logging SMTP traffic typically consumes a great deal of CPU time and disk space.

Figure A

You must weigh the benefits of having such a log against the impact that logging SMTP traffic will have on your server.

The General tab's real power is revealed when you click the Advanced button. The resulting Advanced dialog box is designed to allow you to associate multiple IP addresses with the selected SMTP virtual server. Although this dialog box doesn't make it obvious, you can also apply filters to the SMTP virtual server. To do so, select the existing entry and click the Edit button. The resulting dialog box contains check boxes that you can use to apply sender, recipient, and connection filters.

Keep in mind that this dialog box doesn't allow you to create the filters; it only lets you turn the filters on and off. To create the filters, you must close all of the open dialog boxes and go to a completely different part of the Exchange System Manager.

To create the filters that you can apply to your SMTP virtual server, navigate through the Exchange System Manager to your organization | Global Settings | Message Delivery. Now, right click on the Message Delivery container and select the Properties command from the resulting shortcut menu. When you do, you will see the Message Delivery Properties Sheet.

This properties sheet contains separate tabs for sender filtering, recipient filtering, and connection filtering. Adding items to the individual filters is pretty much self explanatory, so I'll spare you the details, but I do want to briefly review what the individual filters do.

The sender filter allows you to block any inbound messages that claim to be from a specific person. For example, at one of the offices where I used to work, one of the female employees kept receiving harassing E-mail messages from her ex-husband. I was able to block the messages by using a similar type of filter. The Sender Filtering tab also gives you the option of blocking messages with a blank sender. You can also select a check box that prevents the sender from being notified that their message was blocked.

Recipient filtering works very similarly to sender filtering. The idea is that you can block SMTP messages that have been sent to specific recipients. For example, I have seen some organizations that will set up a special Exchange mailbox that corresponds to a resource such as a conference room. The idea is that Exchange's scheduling feature can be used to book the resource. If your organization has such a mailbox, then you probably don't want people sending SMTP mail to it.

Another good example is the Administrator account. In some organizations, the Administrator account for a domain might also have an Exchange mailbox. You certainly don't want people sending E-mail messages to the Administrator account. The Recipient Filtering tab also contains a check box that you can use to filter any recipient who is not listed in the directory.

The connection filtering option works similarly to the sender filtering option. The biggest difference is that while the sender filtering option allows you to block specific E-mail addresses, the connection filtering option allows you to block a message based on its domain or IP address of origination. You also have the option of always accepting E-mail messages from specific IP addresses.

The Access tab

Now that I have talked about the Default SMTP Server Properties Sheet's General tab and how to set up various filters for it, let's move on to the properties sheet's Access tab, shown in Figure B. By far the most important feature on the Access tab is the Relay button. If you click the Relay button, you are taken to the Relay Restrictions dialog box. This dialog box allows you to either allow everyone, except for people that you specify, to relay mail through your server, or you can set it so that nobody can relay mail through your server unless you specifically give them permission to do so. Obviously, you should use the latter option.

Figure B

By far the most important feature on the Access tab is the Relay button.

The Relay Restrictions dialog box also contains a check box that you can use to allow anyone who has successfully authenticated with the server to relay mail through it. There are a couple of different schools of thought on this option. Some people believe that if a user has authenticated, then they are a trusted user and if they need to relay a message then it's OK. In case you are wondering, there are legitimate uses for mail relay.

For example, mobile users are sometimes forced to relay messages when a direct VPN connection is not available. On the flip side, some people believe that you should not even allow authenticated users to relay mail through your server because a spammer might figure out one of your passwords and start relaying mail through your server.

My personal feelings are that if you have a legitimate reason why your users might need to relay mail, then go ahead and allow authenticated users (or specific authenticated users) to relay messages. Just be sure to block message relaying for everyone else.

In the previous paragraph, I talked about allowing authenticated users to relay messages. One thing that you have to consider though is how you will allow users to authenticate. By default, any type of authentication, including clear text, is accepted. I recommend using the Authentication button to lock down the authentication mechanism to prevent anyone from logging in using basic authentication.

The Messages tab

The Message tab, shown in Figure C, is designed to protect your server against deliberate denial of service attacks and against accidental E-mail abuse. The first two check boxes found on this tab allow you to set message and session size limits respectively. You have to use a little common sense when setting these options though.

Figure C

The Message tab is designed to protect your server against deliberate denial of service attacks and against accidental E-mail abuse.

For example, I have known Exchange administrators who set a 4 MB limit on inbound messages. At the same time though, there are sometimes legitimate business needs for sending large E-mail attachments. For example, I'm betting that the next batch of articles that I send my editor will probably be at least 6 MB in size.

The next two options allow you to limit the number of messages per connection and the number of recipients per message. These options allow you to prevent someone from flooding your mail server. For example, you could set limits so that nobody is allowed to send more than 20 messages at a time, and each message must have fewer than 50 recipients. Such restrictions help to prevent denial of service attacks and SPAM.

The next four options deal with non-delivery reports, the Bad Mail folder, and unresolved recipients. I recommend leaving these options blank. Non delivery reports should be used sparingly because non delivery reports can consume a tremendous amount of system resources if spammers try to guess the E-mail addresses of your users (a common technique).

You don't really have to worry about the Bad Mail options either. Microsoft disabled the Bad Mail option in Exchange Server 2003 Service Pack 1 as a way of combating SPAM. The final option on this tab is to enter an E-mail address to forward unresolved messages to. Again, you don't really have to worry about this because messages sent to non resolvable recipients are almost always SPAM.

The Delivery tab

The Delivery tab, shown in Figure D, is mostly unrelated to security. It primarily deals with how often Exchange will attempt to send messages when it's having trouble delivering an E-mail. You will notice in the figure though that there is an Outbound Security button.

Figure D

The Delivery tab deals primarily with message retry intervals.

By default, anyone can send an outbound SMTP message without having to provide any additional authentication credentials. However, the Outbound Security option allows you to require authentication for anyone transmitting an outbound SMTP message. You can choose from basic authentication (the password is sent in clear text) and Integrated Windows Authentication. There is also a check box that you can select to enable TLS encryption.