Data Centers

Level 5 disasters: The worst-case scenario

Mike Talon examines the worst-case-scenario disaster situations and how you can prepare a plan to address them.

"The Worst Case Scenario," or a total loss of all data-center structures and systems, is the last thing any of us wants to think about. However, it is entirely possible that such a disaster could hit your organization, as was shown during September with Hurricanes Katrina and Rita, here in the United States. During this type of disaster, the idea of Disaster Recovery (DR) is going to take on a whole new meaning, as your organization scrambles to correct for everything that has gone wrong in such a short period of time. Major floods, fires, hurricanes, and man-made disasters such as terrorist attacks fall into this category.

Though the largest of organizations will be preparing for these disasters with availability solutions to allow them to fail over quickly to another data center outside the scope of the disaster area, most companies will find that a response to a Level 5 disaster is truly a recovery effort instead of a failover exercise. The vast majority of organizations will not be able to afford or manage DR data centers that lie far enough away from the primary facility to be helpful in this type of disaster, meaning that your DR systems will be impacted by the same event that disrupted service at the primary site. Even if you cannot afford to keep full-fledged systems up and running at another location, you can contract to keep backup tapes and other copies of your data in far-flung locations. Many companies specialize in just such recovery services, allowing you to find one that fits both your needs and your budget. This will allow you to deal with the immediate impact of the event, and then recover your data to new systems from the copies warehoused off-site after they are returned to you by the contractor.

Tips in your inbox
How well can your organization deal with an emergency? The Disaster Recovery newsletter helps you protect your valuable data.
Automatically sign up today!

It is at this level of disaster that you will also have to deal with non-technical issues well before your technology plant can come back online. Level 5 disasters almost always include loss of physical space and—unfortunately—loss of life as well. When your employees are no longer available to enact a DR plan, you will need to act as quickly as possible, given the situation, to find new staffers, train them, and get things up and running again. Also keep in mind the immense psychological impact of these kinds of disasters. Employees have probably just lost their homes, possibly their family members, and friends as well. Attempts to coerce such employees to immediately report back to work is both unfair and in many cases unethical, which could leave some large gaps in your DR efforts. Temporary staff may be available in some cases for you to use in the short term, but for the majority of cases you will simply have to redefine your DR plan to take the extra recovery time into consideration.

The best planning you can do for a Level 5 emergency is to prepare everyone for what they can expect, and hold firm if executives try to make you commit to anything unreasonable. Set up phone chains and other alerting structures ahead of time, get your data out of the scope of potential disasters that may impact your production environment, and be ready to deal with the harsh consequences of a massive disaster. The best you can do is to prepare: Level 5 disasters will find every hole your DR plan has to offer.

Editor's Picks

Free Newsletters, In your Inbox