The future of e-mail archiving

Because of corporate accounting scandals of the last few years and compliance with both old and new laws, IT is under pressure to keep more data for longer periods of time and to manage its entire life cycle. An expert from Ferris Research talks about what to expect in the not-too-distant future in terms of e-mail archiving technology.

The corporate accounting scandals of the last few years spawned new legislation, and increased enforcement of old laws regarding the retention of company data. As a result, IT is under pressure to keep more data for longer periods of time, and compliance with these rules carries greater consequences for the entire life cycle of the data. But many companies are still unclear on just what the legal parameters of e-mail archiving are and what they should be doing to comply with regulations. David Via of Ferris Research, a market and technology research firm specializing in messaging and collaboration, agrees that the parameters are still a little fuzzy because there is yet very little case law or precedent on the books for many of the regulations.

"But a couple of them are becoming more clear," Via says. "For example, SEC 17a-3/4 requires that certain types of customer correspondence (including IM) be retained for up to six years. This has driven securities firms to implement archiving very quickly. The impact of other regulations (for example, Sarbanes-Oxley) on e-mail is less certain. Since SOX is concerned with the integrity of the financial reporting process, it depends on the degree to which e-mail and IM are part of an individual organization's processes."

As compliance issues become clearer, IT needs to start ramping up on new archiving technology and trends. Until recently, they relied on user storage quotas or automated deletion processes to keep message store requirements manageable. But now, according to Via, many administrators are being required to implement mechanisms that not only store large quantities of e-mail, but also provide for searching and retrieval. Fortunately, vendors and service providers have been quick to respond with new archiving tools and enhancements.

In a Ferris Research report, David Via talks about some of the archiving trends that will evolve over the next three to five years:

Mail stores recognized as knowledge repositories

Organizations will have to change the way they look at e-mail archiving. They will have to go from the minimum level of functionality for compliance to leveraging the knowledge contained in the archives. This includes implementing powerful search capabilities and the ability to retrieve messages directly from within the e-mail client applications or from a Web browser. According to Via, since the costs of e-mail archiving solutions are so high, organizations need to realize greater value from it than just avoiding fines.

Standardized data storage

Most organizations at present are using magnetic disks, SANs, and NAS for storage. However, as the quantity of data and length of retention increases, organizations will be moving to removable storage data like optical discs or tape, "sometimes in conjunction with an autoloader or jukebox," according to Via.

Flexibility of capture, classification, and retrieval

Products will have to address complex archiving scenarios by factors other than storage efficiency. Archiving solutions are becoming more sophisticated with vendors developing rules engines that enable more selective capture and classification of e-mail. According to Via, "Improved flexibility of capture will have a significant effect on the quantity of storage required for archiving, as well as reduce overall costs. Improved flexibility of search and retrieval will make it easier to satisfy archiving and reporting requirements, and to use message stores as an organizational knowledge repository."

Greater consideration of "document life cycle"

Archiving has traditionally been concerned with what happens to e-mail at the end of its usefulness. But it will start being concerned with the entire life cycle of the e-mail. One promising concept, according to the Ferris report, is "smart archiving," in which awareness of content life cycle is inherent in the data or metadata, probably using a technology such as XML.

Integration with management frameworks, data stores

IBM and Microsoft plan to use DB/2 and SQL Server as message stores in their upcoming products. And the next major release of Windows will embed SQL Server into the operating system as a component of the Windows Future Storage (WinFS) file system.

Service providers will start offering e-mail archiving

E-mail archiving will become an expensive and complex task to manage. It could prove to be more cost-effective to let outside firms, who are focused only on e-mail archiving, handle the task for you. Via says that finding the best options for e-mail archiving comes down to a few issues:

  • Speed to implement: Outsourced solutions can be in place more quickly.
  • Cost: In-house solutions are going to require an initial investment and on-going support.
  • Security: Is the organization comfortable with e-mail data being stored off-site with an outsourcer?
  • Flexibility: In-house solutions typically offer the greatest power to integrate with existing systems and to customize.
    • As e-mail archiving comes of age, its management will become more complex. Now is the time to find out what's out there in terms of e-mail archiving technologies and services. Many of the e-mail industry's leaders"including David Ferris of Ferris Research"will be on the panels at the INBOX E-mail event in San Jose, CA, on June 1-2. The conference will cover archiving, security, authentication, and delivery improvements of e-mail.

About Toni Bowers

Toni Bowers is Managing Editor of TechRepublic and is the award-winning blogger of the Career Management blog. She has edited newsletters, books, and web sites pertaining to software, IT career, and IT management issues.

Editor's Picks

Free Newsletters, In your Inbox