SolutionBase: Stop spam at your server with the Exchange Intelligent Message Filter

Spam is quickly rendering e-mail useless. You can block spam at your Exchange 2003 server using Microsoft's Intelligent Message Filter. Here's how.

For more Microsoft Exchange server tips, check out TechRepublic's Tech Tips for Exchange Administrators CD-ROM. Packed with more than 100 technical solutions, this tips collection simplifies Exchange 5.5, 2000, and 2003 administration.

Few people would deny that the spam problem has grown to epidemic proportions. While there are a lot of enterprise-level antispam products available for Exchange, most are very expensive and none of them are 100-percent effective. In an effort to turn the tide on the war against spam, Microsoft has released a free antispam component for Exchange Server 2003 called the Intelligent Message Filter.

Some background information

As you probably know, Microsoft owns MSN and Hotmail. For many years now, MSN and Hotmail mailboxes have been favorite targets of spammers, perhaps rivaled only by AOL mailboxes. Because of this, Microsoft needed to do something to rid these mailboxes of the endless assault by spammers to avoid losing customers.

Unfortunately, spam is really hard to define. To paraphrase Supreme Court Justice Stewart Potter, you may not be able to give a hard and fast definition of spam, but you know it when you see it. Because of this simple fact, Microsoft asked thousands of volunteers to identify messages coming into their Hotmail or MSN mailboxes as being either spam or legitimate.

Microsoft then came up with a program that checks roughly half a million different characteristics of inbound messages. What's nice about the program is that it doesn't just look for characteristics of spam; it also looks for characteristics common to legitimate mail. This improves accuracy tremendously over intelligent mail filtering solutions that merely look for characteristics of spam. The software then uses all of the message's characteristics to compute a mathematical probability of whether or not the message is spam. After using this program successfully in Hotmail, Microsoft decided to create a version of it for Exchange called the Intelligent Message Filter.

Acquiring the Intelligent Message Filter

The Intelligent Message Filter is free for owners of Microsoft Exchange Server 2003. You can download it from Microsoft's Exchange 2003 Web site. The download is roughly 9 MB in size.

Before you install the filter

Before I show you how to configure the Intelligent Message Filter, you need to understand that the Intelligent Message Filter works at the SMTP virtual-server level of Exchange. This means two things. First, if you have someone within your office who sends you lots of junk mail, the Intelligent Message Filter won't filter that mail because it's local rather than SMTP based. Second, if you have more than one SMTP virtual server, you will have to configure the Intelligent Message Filter separately for each one.

Installing the Intelligent Message Filter

Begin by opening the ExchangeIMF.MSI file that you downloaded. When you do, Windows will launch the Microsoft Exchange Intelligent Message Filter Installation Wizard. Click Next to bypass the wizard's Welcome screen and you will see the software's end-user license agreement. Accept the license agreement, click Next, and you will be prompted for the components you wish to install.

There are two components to choose from: the Intelligent Message Filter Functionality option, which is the actual Intelligent Message Filter program, and the Management Tools For Intelligent Message Filter option. If this is the first server on which you are installing Intelligent Message Filter, then you should select both options. It is also possible to install the management component onto a machine that's running Windows XP so that you can manage the Intelligent Message Filter without actually having to sit down at the server console.

Make your selections, click Next, and Windows will begin copying the necessary files. When the copy process completes, click Finish to complete the installation.

Determining the gateway threshold

Once the Intelligent Message Filter is installed, you must determine the gateway threshold value. The idea here is that your Exchange Server is acting as a mail gateway. Messages come into the server from the Internet and are placed into user's mailboxes. The idea behind setting the gateway threshold value is that the Intelligent Message Filter assigns a value to every inbound message. The value is based on the likelihood of the message being spam.

This is where the gateway threshold value comes in. If a message's value exceeds the gateway threshold value, the Intelligent Message Filter assumes that the message is spam and doesn't even bother placing the message into the destination mailbox.

The default gateway threshold value is 8, but this value is not suitable for all installations. If the gateway threshold value is set too low, the Intelligent Message Filter may start flagging legitimate mail as spam. If the gateway threshold value is set too high, on the other hand, users' inboxes may be flooded by spam. It's a very fine balancing act, and this is why it's important to find out the appropriate value for your organization based on the mail that you receive rather than simply accepting the defaults.

To figure out the appropriate value for your gateway threshold, you will have to use the Performance Monitor. When you install the Intelligent Message Filter, you are also installing a set of corresponding Performance Monitor counters. The tricky part, however, is that these counters are not readily available. The counters become available only after messages begin passing through the filter. Fortunately, there is a way to have messages pass through the filter without actually taking any action on the messages.

To do so, open the Exchange System Manager and navigate to Global Settings | Message Delivery. After doing so, right-click on Message Delivery and select the Properties command from the resulting shortcut menu. This will cause Exchange to display the Message Delivery Properties sheet. Select the Intelligent Message Filtering tab, then verify that all thresholds are set to a value of 8. You must also verify that the When Blocking Message option is set to No Action, as shown in Figure A.

Figure A

Configure the Intelligent Message Filter to take no action for right now.

Click OK and then navigate through System Manager to Administrative Groups | your administrative group | Servers | your server | Protocols | SMTP | Intelligent Message Filtering. Right-click on the Intelligent Message Filtering option and select the Properties command from the resulting shortcut menu. Select the check box next to the SMTP virtual server for which that you want to enable Intelligent Message Filtering, as shown in Figure B. Click OK, and you should now be able to access the Performance Monitor counters. If not, you may have to reboot your server.

Figure B

You must enable Intelligent Message Filtering for each SMTP virtual server that you want to use it with.

At this point, open the Performance Monitor and remove any existing performance counters by selecting them and clicking the X icon. Next, click the + icon to reveal the Add Counters dialog box. Select the MSExchange Intelligent Message Filter performance object, then select the Total Messages Assigned An SCL Rating Of 0 counter. Click the Add button and repeat the process to add the counters for SCL levels 1 through 9. When you're finished, click Close and then click the icon that formats the data as a bar graph. You should now see an empty graph similar to the one shown in Figure C.

Figure C

This is how Performance Monitor should be configured.

You'll want to wait at least one business day for the Performance Monitor to collect an accurate sampling of data. If your organization doesn't get a lot of e-mail, you may need to wait longer. At any rate, you will eventually have a graph that looks something like the one shown in Figure D.

Figure D

This is what a fairly typical set of results will look like.

In this case, though, Figure D is a mock up. I use a pop3 utility to download all of my e-mail from my ISP to my Exchange Server and, therefore, my server doesn't receive any SMTP mail. Even so, the chart in Figure D shows a fairly typical set of results.

As you look at Figure D, you will notice that there are ten different bars on the chart. The bar on the far left represents the number of received e-mails with an SCL (spam confidence level) of 0. The bar to the far right represents the number of messages with an SCL of 9. If an e-mail message has an SCL of 0, it means that the Intelligent Message Filter is positive that the message is legitimate. Likewise, if the SCL rating is 9, then the Intelligent Message Filter is positive that the message is spam. Messages with SCL ratings below 5 are most likely legitimate mail, while messages with an SCL rating above 5 are most likely spam. This doesn't mean that you should set the gateway threshold value at 5, however.

If you look at Figure D, you will notice that some SCL ratings were much more common than others. Particularly, 6, 7, and 8 were the most common ratings. There was a very sharp rise in mail volume from an SCL value of 5 to an SCL value of 6. Therefore, in this particular case, you would probably want to set the gateway threshold value to 6. The reason is that all messages with an SCL of 6 or higher would be treated as spam at the gateway level. As you can see in the figure, this would eliminate most of the inbound mail. On the other hand, if there had been relatively few messages with an SCL rating of 6, but a lot of messages with an SCL rating of 7, then you would probably want to set the gateway threshold value to 7. The trick is to set the gateway threshold value to the number corresponding to the SCL rating where you see the sharpest rise above seemingly legitimate mail. In this case I picked 6 because there were only about five messages with an SCL of 5, but there were about 40 messages with an SCL of 6.

Now that you know how to figure out the appropriate gateway threshold, it's time to actually set it. To do so, return to the Intelligent Message Filtering tab of the Message Delivery Properties sheet. Next, select the appropriate SCL rating value within the Gateway Blocking Configuration section. Before the gateway will filter any spam though, you will need to change the When Blocking Messages option from No Action to either Archive, Delete, or Reject.

Controlling spam for users

Now that you have set the gateway threshold value, you have gotten rid of most of the spam that's coming into your organization. However, there is still a lot of mail coming in that might or might not be spam. Since there is a possibility that some of this mail might be legitimate, you don't want to have your Exchange Server getting rid of it at the gateway level. Instead, it's better to have the users to make a decision as to whether the mail is legitimate or not.

One way of accomplishing this is to configure the Intelligent Message Filter to move potential spam that has not already been filtered at the gateway level to a user's Junk E-Mail folder within Outlook. To do so, let's look at Figure D one last time. In the figure, you will notice that there is quite a bit of mail that has been assigned an SCL rating of 0 or 1. The number drops off significantly at 2 and climbs again at 3. The graph is a good indication (at least in this case) that SCL levels 3 through 5 are questionable messages that could potentially be spam.

This being the case, we will tell the Intelligent Message Filter to move any messages with an SCL rating of 3 or above into the user's Junk E-mail folder. The messages won't actually be deleted—they are simply being moved to a location in which they will not show up in the user's Inbox, but in which the user is free to review them if necessary. To set this threshold value, return to the Intelligent Message Filtering tab of the Message Delivery Properties sheet and set the Store Junk E-Mail Configuration value to the appropriate level (in this case 3).

Spam control within Outlook

So far we have configured the Intelligent Message Filter to make some educated guesses as to what messages should and should not be classified as spam. Unfortunately, the Intelligent Message Filter is not perfect in its judgment, so it is prudent for users to help the Intelligent Message Filter out a little bit by configuring Outlook to recognize both legitimate mail and spam. For example, I receive a bi-weekly newsletter through e-mail called the Relevant Security News. It's a newsletter packed with information about IT security. Even though this newsletter is very important to me, my spam filter simply sees it as something that was mass mailed, and therefore flags it as spam. To counteract the problem, I set up a whitelist and placed the e-mail address that distributes my newsletter on it.

The idea behind a whitelist is that senders who are on the list never have their messages flagged as spam, regardless of the message content. Likewise, you can also set up a blacklist. Blacklisted sender's messages are always flagged as spam, regardless of whether the message is legitimate or not.

Almost every antispam program has a blacklist/whitelist feature, and this is generally how the feature works. In an Intelligent Message Filtering environment, the blacklist and whitelist work a little bit differently than you might expect. The reason is the gateway filtering option. Remember all of those messages that we configured the Intelligent Message Filter to delete at the gateway level? Those messages will never be compared to a user's blacklist or whitelist, because the blacklist and whitelist are mailbox-level features. When you delete messages at the gateway level, you are deleting them before they can ever even reach the mailbox level.

Because of this, some administrators prefer to set the gateway threshold to a very high level, such as 8 or 9, so that only the most blatantly obvious spam is deleted. This allows more messages to make it to the mailbox level where they can be compared against the user's blacklist and whitelist prior to being moved to the user's Junk Mail folder.

So how do you manage all of those messages that do make it to the mailbox level? The first step is for the users to set up whitelists and blacklists. They can do so by opening Outlook 2003 and selecting the Options command from the Tools menu. When the Options properties sheet appears, the users can click the Junk E-Mail button. Tabs then become available for setting up whitelists and blacklists. In Outlook, these options are referred to as the safe senders list and the blocked senders list. If you happen to have a blacklist or whitelist in another antispam program, Outlook provides a way to import these lists. There is also an option to consider any messages from someone with an entry in the user's Contacts folder as safe. Outlook allows users to place about 2,000 entries on the safe senders list.