Software

Filter mail with powerful procmail


Procmail is perhaps one of the most powerful mail filtering programs available. It plugs in seamlessly with fetchmail and can do an amazing number of things, including simply filtering mail into various mailboxes, forwarding mail to another account, or executing programs.

Procmail is powerful and complex, and can be intimidating to those who have never used it before. However, no other mail filtering support built into an e-mail client can compare to it.

To begin, procmail uses ~/.procmailrc by default. In the interest of organization, and because you can use multiple files to create your rule set, it may be better to create a directory called ~/.procmail/ with the file ~/.procmail/procmail.rc as the primary configuration file and symlink ~/.procmailrc to it. For instance:

$ mkdir .procmail
$ touch .procmail/procmail.rc
$ ln -s .procmail/procmail.rc .procmailrc

A very simple ~/.procmail/procmail.rc may look like:

DATE=`date +%Y-%m`
SHELL=/bin/sh
VERBOSE=off
PMDIR=/home/user/.procmail
LOGFILE=$PMDIR/procmail.log
LOG="
"
MAIL=/home/user/Mail

:0 Wh: msgid.lock
| formail -D 8192 $PMDIR/msgid.cache

:0 c
$MAIL/Archives/incoming-$DATE

INCLUDERC=$PMDIR/deletes.rc
INCLUDERC=$PMDIR/mailinglists.rc

:0
* ^TO_.*user@school.edu
$MAIL/Inbox-school
:0
$MAIL/Inbox

This may look complicated, but it's quite straightforward. The first few lines define various variables, just like one would find in a shell script. Of particular interest are $PMDIR, which defines where the various procmail-related files live (~/.procmail/). Also, $LOGFILE defines where procmail logs its actions. When debugging or starting out, you will want to set $VERBOSE to "on" in order to capture as much information as possible.

The $LOG variable defines the separator between log entries; in this case it inserts a blank line. Finally, $MAIL determines where mailboxes live. Not all of these variables are specific to procmail -- some are variables that are reused in later recipes. The ones specific to procmail are $SHELL, $VERBOSE, $LOGFILE, and $LOG.

The first recipe creates a cache of recently-received Message-ID's of e-mails and weeds out duplicate messages. In this case, the cache is in the file ~/.procmail/msgid.cache and will remain approximately 8KB in size.

The second recipe creates a copy of every single message received and stores it in a by-month dated mailbox. With this recipe, even if procmail does something unexpected, you retain all of your e-mails. With it, you can read and delete e-mails with impunity knowing you can always go back to find a message at any time.

The next two lines set the $INCLUDERC variable, which tells procmail to include the noted files at that point in processing. In other words, after the first two recipes, procmail will proceed through the recipes in ~/.procmail/deletes.rc and then ~/.procmail/mailinglists.rc before proceeding further.

Finally, the last two recipes define final-destination delivery. With the first, if the e-mail is addressed to user@school.edu (be it in To: or CC:), the mail will be filtered into the ~/Mail/Inbox-school mailbox. All other messages that have not met with any other rules get filtered into ~/Mail/Inbox.

Next week, we will look further at what procmail can do.

Delivered each Tuesday, TechRepublic's free Linux NetNote provides tips, articles, and other resources to help you hone your Linux skills. Automatically sign up today!

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

17 comments
Absolutely
Absolutely

So I didn't use it. But, by replacing [i]* ^TO_.*user@school.edu $MAIL/Inbox-school[/i] with " :0 * ^From.*@school.edu | /usr/bin/maildrop " I got my mail fetcher to handoff to Courier, and deliver my mail from my old busted gateway/firewall machine to my primary workstation. [I'm not sure about Vincent's versions of procmail/fetchmail, and this is pretty new to me, but my man pages said nothing about any '_' in those lines of the recipe. IIRC, it didn't work correctly with '_' so I removed it. The only other difference in our syntax, the '|', is probably known to most readers; it takes the output of the command before it and uses that as input to the command following it.]

caebte
caebte

I made a simple webapp at work for the cafeteria - it prompts the user to create a lunch order. What they submitted was a nice html page with their name, items, price, pickup time, etc.. The form submit used a phpmailer function to email the html to a local account I created on the linux webserver. I set up .procmailrc file to first archive the message, then do some clean up, and finally convert it to postscript and print it out. Any order coming in got printed to the network printer in the cafeteria where they could fill the order. (NOTE, the "O" in ":O" is really a zero - if i leave it a zero it gets converted to a smiley.. :0 ) #copy the message to archive directory :O c archive # clean up... :O ic | cd archive && rm -f dummy `ls -t msg.* | sed -e 1,300d` #convert and send to printer :O b | /usr/local/bin/html2ps | lpr -Ptehpcafe # delete the rest :0: /dev/null

jjrowan
jjrowan

I use procmail recipes but often have problems with mail being lost. I have been trying to find a book on procmail but can't find anything. If someone has a link to a site that provides extensive documentation on this tool I would appreciate it. Please don't say to read the man pages on it, I've been there and need more.

Absolutely
Absolutely

that was not [u]supposed[/u] to be an emoticon, but if you just copy it, into a text editor, it will display correctly as a ':' followed by a '0'. Bloody TR! It should not display as a cartoon [u]without[/u] some special "tag"!

techrep
techrep

Yes, many people talking about "man procmail" but many of them forgot to say about "man procmailex" - man page with many usable examples. Maybe this help you.

jlwallen
jlwallen

when i'm using procmail - especially if i'm testing recipes - i always add a backup to the recipe such as: BACKUP=$MAILDIR/backup TODAY=`date +%d-%m-%Y` #no space between the : and the 0 : 0 c: $BACKUP/$TODAY make sure you create the ~/Mail/backup directory of course. put that as the very first recipe and all the mail will back up before it's acted upon.

Jaqui
Jaqui

by using the pre tag for code snippets? [pre] * ^TO_.*user@school.edu $MAIL/Inbox-school[/pre] with [pre] :0 * ^From.*@school.edu | /usr/bin/maildrop[/pre] you only have to use the [ around them :) edit to add: nope, that doesn't work, it still displays the emoticon.

stew
stew

I have the following in my .procmailrc so I can remember the syntax described in a way that's meaningful to me. (I hope it looks good after the post!) ################################################################################ # Recipe format # # :0 [flags] [: [lock-file]] # # # # ----- Flags ------------------------------------------------------------------ # # H the header must match the filter(s)* # B the body must match the filter(s) # D match case-sensitively # A ignore this recipe unless the conditions on the preceding recipe at the same # block scope w/o an "A" or "a" flag matched (AND) # a like "A" but the preceding recipe w/neither the "A" nor "a" flags must have # completed successfully (AND) # E ignore this recipe unless the preceding recipe did not execute (OR) # e ignore this recipe unless the preceding recipe failed (OR) # h pipe the header to the action, which is responsible for delivery # b pipe the body to the action, which is responsible for delivery # f pipe the message through the action making the output of the pipeline the new # message; procmail continues to process the message normally # c delivers according to the current recipe and looks for more recipes # w wait for the filter/program and check the status code: failure means no # filtration # W same as "w" but suppresses the "Program failure" message # i ignore write errors (usually due to an early closed pipe) # r raw mode: don't ensure message ends with an empty line # # * default # # ----- Conditions ------------------------------------------------------------- # # * [scoring] [prefix] regexp|? process|< size|> size # # There can be multiple regexp's; procmail AND's them. # # Prefixes: # # Prefix | Effect on the Remainder of the Condition # ----------+------------------------------------------------------------------ # ! | Negate the condition # $ | Evaluate according to sh(1) double quoted text substitution # | rules, skip leading whitespace, then reparse # ? | Run a command and use its exit code # < | Check whether message length is less than the number given # > | Check whether message length is greater than the number given # e-var ?? | Match against the value of ${e-var} # B ?? | Match against the body (despite contrary flags) # H ?? | Match against the header (despite contrary flags) # BH ?? | Match against the header and body (despite contrary flags) # HB ?? | Match against the header and body (despite contrary flags) # \ | Quotes any of the above at SOL # # Note that there are two "From" fields. The one followed by a colon is that of # the real sender; the one without a following colon is the claimed sender. # # ----- Scoring ---------------------------------------------------------------- # # A score > 0 means to take the given action. # # [-]w^x # # where w is the points to add (or subtract) and x means: # # x | Impact on Score # ----------+----------------------------------------------------------- # 0 | The first matching line adds (subtracts) w # 1 | All matching lines adds (subtracts) w # 0 < x < 1 | Each matching line contributes a decaying amount # x > 1 | Each matching line contributes an increasing amount # x < 0 | "Can be utilised to favour odd or even number of matches." # # described by this equation: # # n n # -- k-1 x - 1 # w * > x = w * ------- # -- x - 1 # k=1 # # Typically, you initialize the score with x = 0 and score the remaining lines # with x = 1. Another approach is to start with x = 1 and score other lines # with x = -1; that means the action will be taken unless one of the regexp's # matches. # # ----- Actions ---------------------------------------------------------------- # # There can be only one action line per recipe, but it can span multiple lines # in the file if you continue it with "\" at EOL. # # Symbol | Action # --------+--------------------------------------------------------------- # ! | Forwards to all specified addresses # | | Starts the program (via $SHELL if the command contains any # | characters in $SHELLMETAS) and pipes the message text to it # e-var=| | As with | but the output is saved in ${e-var}

vdanen
vdanen

DATE=`date +%Y-%m` SHELL=/bin/sh VERBOSE=off PMDIR=/Users/vdanen/.procmail # # delete duplicate messages first # : 0 Wh: msgid.lock | formail -D 8192 $PMDIR/msgid.cache # # keep our archive in dated mboxes # : 0 c $MAIL/Archives/incoming-$DATE The above is how I start my procmailrc. All messages, except dupes, land in Archives/incoming-2007-12, etc. This allows me to delete email with impunity, knowing I can always find an old message. I have about 3 years of email stored this way. I have (older) copies of my procmail recipes on my "mutt on OS X" webpage... it works the same regardless of whether you're using OS X or Linux. http://linsec.ca/Using_mutt_on_OS_X Dang smilies. There is no space between ":" and "0" above.

Absolutely
Absolutely

to spend 20 minutes Photoshopping Tux's background. But, it looked like he was sitting on 'my' face, so I had to black out the default face background image. Y'know, speaking of TR nuisances.

Jaqui
Jaqui

contents should never be parsed for emoticons, it should ALWAYS be sent as is. The idea being it is a designation of content that is to be displayed as typed, possibly code, which should be treated as a security risk and forced to be treated as text only.

Absolutely
Absolutely

I edited that. Foul, -34 Tech Points for 'absolutely.'

Absolutely
Absolutely

So, although I can put a tag around [pre]:0[/pre], if I put it around a longer block of code, the likelihood of some part of it messing with the 'pre' tag rapidly approaches 'unacceptable.'

Jaqui
Jaqui

I'm seeing the emoticons even when it's in the pre tag.

Absolutely
Absolutely

The 'pre' tag is exactly what I used, in my whiny reply to myself. Didn't it work there? What is going on, Jaqui??? '[pre] :0[/pre] * ^From.*@school.edu | /usr/bin/maildrop '

Absolutely
Absolutely

"There can be only one action line per recipe ..." About your text -> emoticon problem, the solution is the "pre" tag for "Verbatim" text. http://articles.techrepublic.com.com/5100-22_11-5272932.html I've tried to tell The Management that since this is an IT site, not a social networking site, that Verbatim should be the default, and emoticons should require extra tagging, or better yet be self-made in Flash or .gif animations or whatever. They don't seem excited about my ideas. [pre]:0[/pre]

stew
stew

FYI: I see an emoticon where the text has a colon followed by a zero.

Editor's Picks