Software

How do I... Archive Outlook e-mail without a PST file?

IT consultant Erik Eckel shows you how to effectively store old e-mail while maintaining accessibility to archived messages.

This article is also available as a TechRepublic download and as a TechRepublic gallery.

Is e-mail getting you down? Do administrative limits keep you pruning precious messages and attachments from your Inbox? Or worse, do you have to import numerous unneeded messages every time you must recover a single e-mail message?

There's a better way.

Organizations and employees, of course, are increasingly dependent upon e-mail. From correspondence with customers and clients to strategic communications with colleagues, vendors, suppliers, and others, e-mail has become a business critical application.

Information technology departments have responded accordingly. Back-end e-mail servers are often well-hardened, run on dedicated systems, and feature intricate remote or offsite backups. But an age-old problem remains: how to effectively store old e-mail while maintaining accessibility to those messages.

The PST file

Microsoft's answer has been the PST (Personal Storage File) file and a complex archiving dance. Redmond uses the PST file to store Calendar items, e-mail, Journal entries, and Tasks on local Windows workstations. Users are responsible for proper archiving (which requires specifying archive periods, file locations, folder and subfolder locations, etc.).

The PST file format, meanwhile, is used with the Microsoft Exchange Client, Windows Messaging, and Microsoft Outlook 97/98/2000/2002. Microsoft Outlook 2003 introduced new functionality with support for Unicode data (as opposed to supporting only ANSI text encoding), meaning older versions of Microsoft Outlook encounter trouble when trying to access or open PST files from newer editions.

The Outlook 2007 PST file, like its Outlook 2003 counterpart, is not compatible with Outlook versions 2002 and earlier. However, Outlook 2003 and 2007 can open PST files created by earlier Outlook versions.

The PST files themselves are notorious for corruption. The storage format has proven so delicate (but critical) that numerous repair utilities -- the Inbox Repair Tool/Scanpst.exe, EasyRecovery Email Repair, EasyRecoveryFileRepair, Search and Recover, Stellar Phoenix Mailbox Professional PST Recovery Software, Repair PST, etc. -- exist for attempting to correct common corruption issues.

The file format has another significant drawback: size. Outlook 2000's PST file was limited to 1.933GB. Routinely in the field, I encounter Outlook PST files in excess of four and five gigabytes.

Outlook 2003 increased the size limitation to 30GB, but the larger the file often the slower the performance. Worse, when PST files do fail irrecoverably, the larger the file size the greater the loss from lost messages, reports, attachments and other information.

Thankfully, there's an effective alternative to archiving Outlook e-mail using only PST files.

Make folders instead

Just use folders.

Within My Documents on your desktop, create a folder titled Email (Figure A). Within that Email folder create subfolders (Figure B) for each folder you've created in Outlook (beneath your Inbox). Within those subfolders, create monthly or quarterly folders (Figure C) within which you place copies (just drag-and-drop them en masse) of every e-mail message you store within Microsoft Outlook. You can also create folders for Sent Mail and, if you're truly into backing up all your e-mail, Deleted Items as well.

Figure A

To begin using the folder system to archive e-mail, create an Email folder within My Documents.

Figure B

Within the Email folder, create subfolders for each folder you maintain within your Outlook Inbox.

Figure C

Within each subfolder, create relevant subfolders (to mimic the structure of your Outlook Inbox).

Once you've copied messages to the Email folder, you can delete them from Outlook. Thus you receive an added benefit. Outlook will open and operate more smoothly and consistently, as you'll be working with a much smaller PST file (which will power only active e-mail messages, tasks, contacts, and calendar items that haven't been copied to the Email folder and deleted from Outlook). Plus, maintaining a small PST file will keep you out of trouble with the Exchange administrator (who likely implements and maintains strict mailbox size limits).

But there's more. What is the biggest advantage?

Whenever you need to revisit a specific e-mail message or (as is increasingly the case for many) a corresponding attachment, all you need to do is locate the proper folder, retrieve your e-mail, and you're done. There's no need to complete cumbersome import/export processes (that also require numerous other unnecessary e-mail messages and attachments be recovered).

Best of all, you don't even need to remember in which folder you placed an e-mail message. As mentioned previously, the rise of popular desktop search utilities (including the Windows Desktop Search engine built into the new Windows Vista operating system) makes easy work of locating an old e-mail message and/or attachment.

You can even archive messages by month, quarter, or any other period (Figure D). Just create the required folders/subfolder structures (Figure E).

Figure D

Within each Outlook Inbox subfolder, create monthly or quarterly folders to ease how often you need to manually copy e-mail. Breaking messages into dated folders also greatly simplifies recovering specific e-mail messages (although desktop search engines typically fulfill that function now).

Figure E

Individual e-mail messages (with any corresponding attachments) are then placed directly within these folders.

You can even tap Outlook's Task or Calendar functionality to remind yourself you need to archive e-mail. Whenever the reminder appears, copy e-mail messages to your new folder structure following these steps:

  1. Open (within Outlook) each e-mail folder you wish to back up.
  2. Click Edit.
  3. Click Select All.
  4. Click Edit.
  5. Click Copy.
  6. Open the Email subfolder where you wish to archive the e-mail messages.
  7. Select Edit from the Windows Explorer or My Computer menu bar.
  8. Click Paste. The messages will be copied from Outlook to the new folder on your hard disk.

Manage identical messages

When employing the folder system to archive Outlook e-mail without a PST file, there's one item you need to watch. When you copy an e-mail message from Outlook to another folder on your hard drive, Windows creates the file using the message's Subject Line as the filename (adding the .MSG file extension). If you copy multiple e-mail messages at once that have the same Subject Line, Windows simply appends numerals within parentheses to the end of the identical filenames, thereby creating unique filenames for each e-mail message. It works great.

For example, an e-mail message with the filename Test Message will be copied to a folder within My Documents as Test Message.msg. If two or more messages within Outlook share the same Subject Line, Windows simply makes each filename unique by adding numerals within parentheses to the end of the Subject Line: Test Message (1).msg, Test Message (2).msg, Test Message (3).msg, etc.

If, however, you later add an e-mail message to a Windows folder that already holds an e-mail message having an identical Subject Line, Windows will present the Confirm File Replace dialog box (Figure F).

Figure F

The Confirm File Replace dialog box helps prevent accidentally overwriting existing messages.

Be careful in such cases to not accidentally overwrite an existing message. Instead, change the subject line of the existing messages before copying the Outlook e-mails with the identical subjects or, better yet, just create a new folder for the troublesome messages. By manually copying entire folders monthly or quarterly, this problem rarely occurs, however, as all Outlook messages are copied en masse, thereby enabling Windows to automatically create unique filenames.

Find what you need

In the past, just copying e-mail messages to document folders didn't work well, as trying to locate a single e-mail message within thousands of others proved frustrating. With the rise of Google Desktop Search, Windows Desktop Search, and Yahoo Desktop Search, not to mention the fact that Windows Desktop Search is built into Windows Vista by default, locating individual e-mail messages usually requires only that you type a keyword or two within ever-present search boxes (Figures G, H, and I).

Figure G

Using Google Desktop Search, locating specific e-mail messages is a breeze.

Figure H

You can quickly drill down within e-mail search results using Google Desktop Search. Better yet, Google's search provides you with a preview of each message. To access the messages themselves, you need only click the provided hotlink.

Figure I

Using Google Desktop Search, you can open e-mail messages directly within Internet Explorer, should you wish.

Using various available desktop search tools, you can elect to read e-mail message contents within Internet Explorer or open the actual e-mail message (and corresponding attachments) itself (Figure J).

Figure J

Within Windows Vista, the integrated Windows Desktop Search is very quick to locate and identify specific e-mail messages.

Simple, secure backups

Using an e-mail folder archive system, e-mail backups are greatly simplified, too. No longer do you need to worry about PST maximum file sizes, incompatibilities, or corruption issues. Just backup the Email folder (and all its subfolders) to your backup medium of choice and your e-mail is protected.

Restoring e-mail is equally straightforward. Just copy the Email folder to a PC or server, and messages (and their attachments) are almost instantly available (without the requirement of importing archive files and specifying new folder/file locations within an existing Outlook mailbox).

If you're worried about security, apply the appropriate NTFS permissions to the Email folder. If you're particularly paranoid about securing your manual e-mail archive, you can also elect to apply Encrypting File System protections to the Email folder. To do so:

  1. Right-click the Email folder.
  2. Select Properties.
  3. Highlight the General tab.
  4. Click the Advanced button.
  5. Check the Encrypt Contents To Secure Data checkbox.
  6. Click OK.

Summary

Microsoft's PST file format provides one method of archiving e-mail. Creating a folder backup structure, however, simplifies the process and encourages improved Outlook performance. Mated with any of the popular desktop search engines, archiving e-mail using simple folders helps eliminate many of the headaches associated with more cumbersome PST files.

88 comments
Nicholastre
Nicholastre

Horrible "solution"


This is not a "solution" at all, and your business users should give back the money you stole from them

Nicholastre
Nicholastre

What a horrible Solution... you should be ashamed of yourself as a tech person... sorry Help Desk operator for suggesting something like this.

Grace1Jr
Grace1Jr

This was linked from a business class website and for that situation it is a horrible idea, but if you have the freedom to install google desktop on the computer in question it must not be a business situation.

bobby.aviles
bobby.aviles

I use this method at work at my help desk in N.Y.C. This is a great "SOLID" tip! You can use this method for your contact list and notes in Outlook 03 & Outlook 07. Try it. Also try it on a external drive or Thumb Drive. ;)

lstickdo
lstickdo

This approach looks great to me. Has anyone figured out a way to automate the process? Like, having a login script automatically save the contents of some (or all) mail folders to a shared drive? Anyone tried this? Thanks, Lisa

rmohan80
rmohan80

The figures G, H & I are wrong. ONce the email is deleted from Outlook, it doesn't show as an email and worse, you cannot open it as it will throw an error.

mixalis
mixalis

so the advice is to get rid of the software which makes mail management easier and go back to treating everything as a file. so you spend all day categorising things ! forget email then, grab pen and paper and get back to basics !

jturner
jturner

My question is once you take it out of the exchange server does it still qualify under email retention policy or document retention policy?? I believe it now becomes covered under the document retention policy.

bulk
bulk

From experience, I have found that users' mailbox sizes are inflated mainly by the number and size of attachments. Leaving aside the issue of if those files should be attached as individual copies to each recipient or somehow referenced on a central server (Sharepoint anyone?), I've had great success with "attachment save" add-ins for Outlook. There are several of these available from different vendors that seem to work in a similar fashion. They automatically remove any attachments found and replace the attachment with a hyperlink. The attachments are written to a folder of your choice, locally or on a server. The folder structure mimics the folder structure in your Outlook file and you don't have to create it manually. The remaining mail bodies are much smaller, all the search tools will still find whatever you are looking for, it's easy to point the attachments at a network drive if your company is concerned about the security of those attachments and you can configure offline files and folders to make them available to offline mobile users too. This means fewer PST files as users hit their mailbox limits far less quickly. RS

Feral
Feral

Note that in Windows XP, by default the search feature does Not do a text search on .msg files which can make it very difficult to find specific messages when archived this way. To remedy this see http://support.microsoft.com/kb/309173.

bit.68
bit.68

This article is excellent and even better than other articles posted on this web site. Keep going ....

vcent_lim
vcent_lim

Nice tip. But unfortunately it requires lots of steps for those non-IT proficient people. This will benefit those who has large email size, but it will also prove a daunting task to educate those people to use this method. Main con is the extra step to move the messages out manually otherwise this is a good method to trim down pst size. Thanks.

charles
charles

Did a quick browse on the internet and downloaded ABC Amber Outlook Converter. Based on what it can do, it appears to be the solution that everyone is looking for. Take Care. Here is the URL --http://www.processtext.com/abcoutlk.html

microtech
microtech

After selecting the emails I want to copy, I find that the option to "Copy" them is greyed out. The version of Vista I am running is Home Premium. Graham

eharris
eharris

I considered the approach of this article when developing my archiving stragegy without PSTs. Copying the messages locally to your hard drive is an administrative nightmare especially when you become subject to litigation and you actually have to search all of those hard drives for messages. Here's what I did... I created a "public folder" in Exchange for one of my users, then locked it down so that only they could see it or access it. The "public folder" then becomes a personal storage location. I then opened it up so that I could administratively manage it. When the user archives their messages, they simply copy them to their personal "public folder". All email stays in the email server. It's secure and completely searchable. As the Administrator, I see everyone's personal "Public Folder". You as the end user only see your personal folder plus the regular public folders.

JFTierney.KJMC.CIS
JFTierney.KJMC.CIS

I hope anyone who uses this has either a very secure workplace, or never has any information that should not be shared. We are trying to get away from any locally stored files because of the risks of exposing HIPAA or finacial data. This method make all the protected data just as portable. Laptops are a big risk because they are easily stolen, but desktops in an unoccupied office can also go missing. Any locally stored email opens individuals and corporate entities to risk. Unless you are one hundred percent sure that the stored information is secure. Think twice about storing email locally.

Louis Lategan
Louis Lategan

This is nice, but how can I easily see the time of all the messages received in this folder structure?

Gis Bun
Gis Bun

While this may solve some things, it may add others. First all these small files add up to plenty of disk space - even in NTFS. If you aren't [or don't want to] using a desktop search, it doesn't help. SAving this quarterly isn't helpfull either. The MSG files are uncompressed. Here's a better way. Continue to use PST files but in smaller groupings. IE - One for personal Email, one for project related items, one for expense reports, etc. Outlook can add many PST files. This keeps the PST sizes fairly small and allows growth. It uses less disk space and are managable. You can search within Outlook. So no need for extra software. It may take a second or two longer to open Outlook [depending on the number of PST files]. As for backing up, use XP's "Offline Files" and synchronize.

TheGooch1
TheGooch1

My company disables ( via GPO, I would guess ) the ability to store company email in PST files. At the same time, they have continuously decreased the mailbox sizes and added a 2-week email retention time( eg email gets automatically deleted after 2-weeks ). So, I need something like this, but automated. There is definitely too much work involved using the article's method.

herlizness
herlizness

I wouldn't allow this in a business; these days almost every lawsuit involves an extensive discovery request for emails; if they're spread out all over the file system it's a safe bet judges are going to permit inspection of the entire disk ... and if you can't find emails .. the short story is it WILL hurt you all that said, it's a neat "trick" .. I didn't know you could drag an Outlook message into the file system ..

Erik Eckel
Erik Eckel

I would estimate that 90% of the business systems I see have Google Desktop deployed.

Erik Eckel
Erik Eckel

I've been using this archive method for years, and I've deployed it for clients as well. The screen shots were taken from the actual process. Is it possible you encountered some type of corruption or missed a step?

Roger.hill
Roger.hill

Leave the mail on the exchange server. PST files are incredibly inefficient at storing mail. Assoon as you start using them, you lose Single Instance Storage. A 1 meg .doc file sent to 200 pst files occupies at least 200 meg (and likely more because you'll want to back them up, and then you'll archive them to tape). A 1 meg .doc file kept on the Exchange server occupies 1 meg. Outlook Web Access can't access the contents of a PST file. They're just not cost effective. 200 users, each spending a minute a day moving mail to PST files, backing PST files up, repairing them, etc., will use 1000 minutes a week. At $6 an hour this will cost $100 a week. For $50, I can buy a 250 gig hard drive, so for a minute per day per user, I can mirror 250 gigs of data. Of course, if your associates earn more than $6 an hour, or spend more than a minute a day... Okay, the case is not quite that simple, but you get the drift.

Erik Eckel
Erik Eckel

Users don't spend all day categorizing things. My recommendation is to archive, at most, quarterly. The process usually requires ten minutes and proves much easier than trying to load a 2GB PST file (that often fails from corruption anyway) any time you need to access a single archived e-mail.

Erik Eckel
Erik Eckel

I appreciate the kudos. I've found this method the simplest bet that works very well for many small business clients. Obviously, in an enterprise environment it's not always going to work best, but for many it's proven to be a perfect, no-cost solution that's made life much easier.

M.R.
M.R.

Drag and drop isn't a complete solution. I would agree that on a business scale something like Gfi provides a common storage location with security. I have used something like ABC Amber for individual archiving called SMT Archiver. I also export to a PST file. I don't trust the PST or MSG files to not be corrupted or to be compat. with a later ver. of Outlook.

chris
chris

I agree with all the posts that say storing email locally is just creating more issues, either from a security, legal or administrative point of view. Thats why we decided to use GFI Mail Archiver (currently on Version 4). It sits on a server running along side Exchange and uses the journalling ability within exchange to archive emails into either it's own database or a preexisting SQL box/installation. It needs very little maintenance and users can search, access, restore emails from within Outlook. It runs as a web application so can be accessable without using Outlook too. Neat features are: Automatic rollover to archive store at preset times e.g. quarterly, decent amount of security options, saved searches amongst others. GFI do a free trail and through resellers can be very competative with pricing even when a large number of mailboxes are involved. Good luck with your archiving....

TheGooch1
TheGooch1

It doesn't matter to me. The company made the idiotic decision to disallow pst's and limit my mailbox size and mail retention limits so that I cannot store more than a week's worth of email. If they get fined, so be it.

ParysNET Solutions
ParysNET Solutions

We use PST's saved to local desktop then use the PST backup tool to make a copy to the server every evening ready for the backup process. We tried using PST's centrally stored on the server but the process of opening them in the morning across 25 desktops ground the network to a halt with some the PST's being over 1Gb in size. Any solutions to this problem would be gratefully recieved from a security perspective to a useability one...

twoclones
twoclones

More than a lot of work, I'm probably the only person in this company who could do it. I'd love to see an easy way to backup email for everyone on my network automatically a nd without PST files.

TheGooch1
TheGooch1

Yep, too much work for me to even try it out. :(

barryt
barryt

My experience is that the date and time changes when you drag an email into an Explorer folder. The new date and time tells you when it was dragged not when the email was sent/received! This is a big disadvantage of this method of email archiving - I'd love to know of a way round it.

AtCollege
AtCollege

Maybe I am missing something, but I thought the whole point of the article was to avoid Archiving and PSTs at they get corrupted easily. And now I am reading about PSTs and Archiving. If you create the folder system on a network drive, and delete the messages from Outlook that you have copied, then the folder system is backed up and you won't loose anything. The only problem is that if you are not in the office, you would have to VPN into the network or use Remote Desktop to get to the network folders. If this is the way to go, it would solve a big problem for some of our users who have years of email messages with their attachments. We have to keep upping their mailbox size. Our users are Outlook Exchange Clients. Anyone out their implementing this system?

scbuzz
scbuzz

As of about 2 months ago, we (all employees) were notified that we would no longer be able to save email in PST files. All of it had to stay in the inbox (or in subfolders under the Inbox). We could still access any existing PST files, but no new ones could be created nor anything added to existing PST files. This pronouncement came from the office of General Council so it would appear this is being done for purely legal (i.e. Sarbanes-Oxley CYA) reasons.

Neon Samurai
Neon Samurai

I'm up to about five .pst at home and six or seven at work. Objects remain in Outlook with all it's management features but are contained in smaller manageable files outside of the Exchange server.

JFTierney.KJMC.CIS
JFTierney.KJMC.CIS

We have a vendor solution we are possibly implementing last quarter this year, first quarter next. Mimosa Systems is a centralized email mamagement and archive system that will piggyback off of our Exchange server. No local folders, PST, or OST. There is an added advantage in that regardless of how many recipients there are, the foot print is only on email, not one for each.

M.R.
M.R.

I have 'allowed' a handful of users to do this but with warnings. As you have mentioned it would only be usable over VPN when remote. With the size it's likely to grow to this would probably not work. You can set "Offline Files" sync to allow PST files but it may not perform the syncs correctly. PSTs may be fine for none chritical info but they are not a monitored database like on the server. Corruption, size limits, version changes, antivirus, failed backups due to open files, recovery of individual emails, etc. None of these problems have been an issue for us but we only use it for a few users. The MSG files could also have version issues but do eliminate many of the other problems with PST. Personaly though I think the renaming issue would create a lot of trouble and losing the Outlook interface and having to use destop search isn't worth it.

Michael Jay
Michael Jay

Keep the pst files small, under 600 meg so that backing them up to cd's is not a problem. With the larger size pst's in 2003 keep them under 4 gig for backup to dvd archive storage.

cwr64
cwr64

Multiple PST's solve the max size problem for the older versions of Outlook, but I don't expect it would help with the decreased performace caused by having thousands of emails in the PST's. I don't use a desktop search tool, but I do use "Lookout" (available from Microsoft's Office downloads) and with Lookout I can search all my thousands of emails for multiple text strings - in less than 1 second. Lookout has been a huge productivity tool for me and others. It's free.

M.R.
M.R.

I use PSTs for archive to CD but I don't think they should be depended on for regular storage of needed records. I've read many articles about corrupt PSTs. AV can mess them up. Offline files isn't supposed to be used with them. Even when I archive I sometimes run into problems with Outlook releasing the file so it can be cleanly burned to CD.

Schulber75
Schulber75

But can you search for the new file format of office 2007?? I could not! I think that is a big problem you have with Lookout!

mort
mort

I just downloaded Lookout 1.3 from the Geeks website.

rmohan80
rmohan80

Your system will slow down to a crawl.. Please i beg you to uninstall windows desktop search ASAP. I installed it and have been troubleshooting it for the past few hours and have finally gotten it uninstalled.. infact don't use any MS product

chris1955
chris1955

Use the MS Desktop Search. The Google desktop search program slows the system to a crawl. The MS version is pretty quick and I have found it to be better at the number of hits.

ramicitrom
ramicitrom

MS bought out the company that produced LookOut, and incorporated the technology into their MS Desktop search tool. Google Desktop Search has similar functionality. You just need to set it up to index email - it's not default like MS's tool.

Neon Samurai
Neon Samurai

I was really trying to avoid coming close to a pun on that one but what can you do. My PST files are setup iwth specific folder tries similar to all my drives so I generally navigate to the correct grouping folder then let the machine run a quick search on the smaller collection of email if I can't pick it from the list myself. I'll have to have a go at LookOut though; it sounds interesting. It sounds like it's preindexing the folders so it simply searches it's index while hiding the actual data collection for the index in the background.