Data Centers

Four reasons to validate your backup processes

When is the last time you asked WHY you perform backups? Those tapes sitting in off-site storage might just cause you and the other members of IS -- as well as Legal and Internal Audit -- more pain than you realize.

Backups? I can hear it now. "Not another security pundit talking at me about why I should do something I already do…" But that's not what this is about. When is the last time you sat down with other members of the IS team to discuss WHY you perform backups. Those tapes sitting in off-site storage might just cause you and the other members of IS—as well as Legal and Internal Audit—more pain than you realize.

The challenge

I've been involved in many recovery activities over the past 26 years. In many instances, we found we needed to change our backup processes so we didn't expend an extraordinary amount of resources, or risk sanctions because we were unable to recover discoverable data. One truth that has penetrated my gray matter after all these years is we can always do better anticipating and testing for the types of requests coming in from internal and external sources.

The activities that provide the most valuable information about backup process issues are proper data backup design and periodic testing. Both should be based on expected restoration scenarios. The following four scenarios seem the most common:

  1. Recovering failed systems
  2. Replying to litigation discovery requests
  3. Recovering from user or programmer error
  4. Ensuring data integrity
Recovering databases for failed systems

When we talk about backups, most of us immediately think system recovery. Initial backup solution configurations usually focus on this aspect of data recovery, planning for the day when systems fail. The reason we create these backups, usually on tape, is to recover from catastrophic events. However, many organizations keep tapes in off-site storage for years. Regulatory requirements (e.g. payroll records retention) are often used as an excuse. However, information you might have to provide to auditors or government agencies should be easily accessible. Instead of letting the bits deteriorate over time on forgotten tapes on dusty shelves, consider a better data archival system—one that uses inexpensive near line storage (e.g. cheap magnetic or optical disk) and contains only the information you absolutely need to meet legal requirements.

Further, you need to ensure quick recovery when the time comes to rebuild critical systems. The amount of negative impact on a business from a server or data center failure is directly proportional to how long it takes to recover. Recovery solutions must restore critical systems before the maximum tolerable downtime is reached. Innovative approaches can even allow restoration of environments to different hardware platforms. (See Review of Acronis True Image for Microsoft Windows SBS.)

Another problem caused by most DR backups is the tendency to copy everything to one or two sets of tapes. So in deciding to keep tapes indefinitely, you open yourself to issues related to the number two reason tapes are pulled and placed back into tape drives.

Replying to litigation discovery requests

Similar to the security adage, "You don't have to secure what you don't store," you don't have to produce for discovery what you don't have. Information stored on tape or other backup media is potentially subject to discovery. Backup processes should conform to company records retention and legal hold policies. What to store, how long, and on what media are decisions based on regulatory constraints and the potential for litigation. Information destroyed in the normal course of business, in compliance with records retention policies, is not subject to discovery. And you don't have to incur six-figure costs trying to extract if from outdated or poorly indexed backups intended for DR.

If your records retention policies call for saving several years worth of messages, word processing documents, spreadsheets, PDFs, etc., be sure to evaluate the best way to search for and recover this information. Design your electronic archive solutions accordingly. Consider solutions that index information based on content, protect documents and messages found to be relevant to legal hold via integrated search capabilities, and can age with the need to get to the information. In other words, the technology used should not be obsolete and unusable when the time comes to use it.

Recovering from programmer or user error

If I told you that during my 14 years as a programmer I never caused database issues in a production environment, would you believe me? Would you believe anyone who told you? If you answered yes, you haven't been in the IT business very long.

Programmers and users occasionally whack (technical term) databases, resulting in system failure or questionable data integrity. Data restore solutions must allow "surgical recovery," enabling an administrator to quickly search for and restore only what is needed. Recovering small amounts of easily located information reduces recovery time and downtime costs.

Ensuring data integrity

This recovery category is similar to programmer/user error, covering anything else that might taint data integrity. Again, surgical recovery methods should be available to administrators to restore confidence in the data without severe financial business ramifications.


Backup/Recovery plan maintenance

A backup solution is a potential business vulnerability unless it's been tested and the rough spots smoothed. Regular testing against recovery scenarios defined during the design activities is the final step in ensuring recovery processes actually work as expected. Further, review system modifications, changes to retention policies, or shifts in the legal climate to ensure backup and recovery design remains acceptable.

About Tom Olzak

Tom is a security researcher for the InfoSec Institute and an IT professional with over 30 years of experience. He has written three books, Just Enough Security, Microsoft Virtualization, and Enterprise Security: A Practitioner's Guide (to be publish...

Editor's Picks

Free Newsletters, In your Inbox