Software optimize

Why and how you should start using offlineIMAP now

Marco Fioretti explains how offlineIMAP helps you back up selected mailboxes so that you always have a secure copy that you can access on a different computer.

Email may be unglamourous these days, but remains THE most portable, most flexible and most widely supported, text-based online communication system available. Thanks to the IMAP protocol, you can access your email from any computer with any decent email client or browser. Email also is the only channel that is suitable, or allowed, in many work-related situations.

OfflineIMAP makes a really crucial task easy

Of course, since email is so critical, you really don't want to lose your messages. It is crucial to always keep an up to date copy of all your mailboxes, or at least those with the messages you really want or need to preserve, on a different computer. Free Software like OfflineIMAP makes such backups a breeze.

Why do we need a dedicated tool for email backups?

At the file system level, IMAP mailboxes are nothing more than standard directories containing one file per email message (I'm skipping some implementation details here, as they're irrelevant for the topic of this post). If that's the case, you may ask, couldn't we use Rsync for incremental backups of those remote folders? Well, no. Having an IMAP account and password does NOT mean you can log into the IMAP server with rsync, using the same credentials (if you can, that server has a huge security hole!). Besides, IMAP servers only understand... IMAP, not the protocols that rsync uses. That's why you need a separate tool to back up remote mailboxes.

You can start using OfflineIMAP in a minute...

Installing OfflineIMAP usually takes just a few clicks, since it is already packaged for most distributions.

The package has several uses (if you want more posts about it, just ask!). Here, for simplicity, I only cover what I consider the first and most important one, that is, automatic backups of one IMAP account. Setting up this task is as simple as saving the template below as $HOME/.offlineimap.rc and replacing all the uppercase strings with proper values for your account(s):

  [general]
  accounts = GENERAL_ACCOUNTS_NAME
  [Account ACCOUNT_NAME]
  localrepository = Local
  remoterepository = Remote
  [Repository Local]
  type = Maildir
  localfolders = /SOME/FOLDER/ON/YOUR/COMPUTER
  [Repository Remote]
  type = IMAP
  remotehost = IMAP_SERVER_NAME
  remoteuser = USERNAME
  remotepass = THEPASSWORD

Worried about writing the password in a file? I'll deal with that in a minute. For the moment, the configuration above is all OfflineIMAP needs to make and keep in sync a complete copy of all your remote mailboxes on IMAP_SERVER_NAME inside /SOME/FOLDER/ON/YOUR/COMPUTER, in maildir format. Thanks to that file, every time you'll type offlineimap at a prompt, or run it as a cron job, something like this will happen:

  #> offlineimap
   ***** Processing account marco
   Copying folder structure from IMAP to Maildir
   Establishing connection to <my IMAP server>.
  Folder sync [marco]:
  ...
   Syncing archive.2012.10: IMAP -> Maildir
   Adding flag S to 1 messages on .2012.09
  Copy message 343 from .2012.10:
   Copy message 343 IMAP[archive.2012.10] -> Maildir[.2012.10]
  Copy message 344 from .2012.10:
   Copy message 344 IMAP[archive.2012.10] -> Maildir[.2012.10]

...but always start with a Dry Run!

Starting from version 6.5, OfflineIMAP has a Dry Run option that just tells you what the tool would do if you let it run without restrictions:

  #>> offlineimap --dry-run
   [DRYRUN] Creating folder .2012.07[Local]
   [DRYRUN] Creating folder .2012.06[Local]
   [DRYRUN] Creating folder ...

If your distribution doesn't offer that version, make manual backup first, just to be sure you don't lose anything.

OfflineIMAP works on Gmail too!

This alone would be enough of a reason to install OfflineIMAP and use it regularly. Gmail works so well for you that you refuse using any other email client? Fine. But never trust anybody else, including Google, as the only keeper of your email. Like it or not, if they lose something or just decide to kick you out, you're toast. Be prepared. As far as offlineIMAP is concerned, this means:
  • using as remoteuser your complete Gmail address
  • specifying Gmail as the type of remote repository
  • follow the other instructions here for authentication and folder filtering

One interface for every need

OfflineIMAP is very flexible also because it has several user interfaces, all character-based. One, Blinkenlights, is designed to be "fun to watch, and informative". Others are only meant for debugging or interaction with other software. Two interfaces (Basic and Quiet) are optimized for cron jobs. You can specify which interface to use with the -ui switch.

Six ways to handle your password

Back to security now. As explained in the manual, you may also type the password at the prompt each time, put it in several other files or even write a Python function that generates it on the fly. The method to use partly depends on which interface you chose. The Basic one, for example, cannot read input from the keyboard. A good compromise may be to run offlineIMAP as a root cron job, with the password in a file readable only by root.

Organize and rename your folders!

Conserving too many messages may, in practice, turn out to be as useful as conserving none of them. Luckily, offlineIMAP has lots of options to sync only the folders with certain names. Using those filters is very simple, if you handle email in a way that leaves all the messages to be preserved in mailboxes with distinctive names.

Personally, I copy all the messages I want to keep in a mailbox on my IMAP server called archive.YEAR.MONTH (*). With such a structure, these three extra lines in my Remote repository configuration...

  1 folderfilter = lambda folder: re.search('(^archive)', folder)
  2 createfolders = False
  3 nametrans = lambda folder: re.sub('^archive', '', folder)

...are all I need to have offlineIMAP copy on my desktop, without the prefix (line 3), all and only the remote mailboxes whose names start with "archive" (line 1). Simple, isn't it?

(*) yes, I also set up my email client to handle all the details automatically, but that is a (simple) different story)

About

Marco Fioretti is a freelance writer and teacher whose work focuses on the impact of open digital technologies on education, ethics, civil rights, and environmental issues.

2 comments
Neon Samurai
Neon Samurai

This is the one that's boggled my mind; why do IMAP servers not use a standard IMAP protocol? If a server speaks HTTP then it can communicate with an HTTP client. If a server speaks SFTP then it can communicate with an SFTP client. But it's ok that IMAP compatibility is a train-wreck? For example, eGroupware has an IMAP client built in so your calendar/pim webapp does your email too. Makes sense right? Oh, but eGroupware can only speak a few different versions of IMAP so you'll be limited to one of those mail servers.. because someone didn't think IMAP should be standard across IMAP servers. (eGroupware front end with Citadel managing mail users/mailboxes in the background.. sublime.. if it where possible) Now, I gotta go see if imapOffline is better than my current backup option. :D

mfioretti
mfioretti

Hi Neon Samurai as said in the comment yes, I understand what you are talking about, and share your pain. IMAP in several senses is still stuck at where HTML was 10+ years ago. Eventually, we all hope, things will improve. Oh, and do let us know if offlineIMAP is better than your current option (what is that option, by the way? Really curious)