“The Worst Case Scenario,” or a total loss of all
data-center structures and systems, is the last thing any of us wants to think
about. However, it is entirely possible that such a disaster could hit your
organization, as was shown during September with Hurricanes Katrina and
Rita, here in the United States. During this type of disaster, the idea of
Disaster Recovery (DR) is going to take on a whole new meaning, as your
organization scrambles to correct for everything that has gone wrong in such a
short period of time. Major floods,
fires, hurricanes, and man-made disasters such as terrorist attacks fall
into this category.
Though the largest of organizations will be preparing for
these disasters with availability solutions to allow them to fail over quickly
to another data center outside the scope of the disaster area, most companies
will find that a response to a Level 5 disaster is truly a recovery effort
instead of a failover exercise. The vast majority of organizations will not be
able to afford or manage DR data centers that lie far enough away from the
primary facility to be helpful in this type of disaster, meaning that your DR
systems will be impacted by the same event that disrupted service
at the primary site. Even if you cannot afford to keep full-fledged systems up
and running at another location, you can contract to keep backup tapes
and other copies of your data in far-flung locations. Many companies specialize
in just such recovery services, allowing you to find one that fits both your
needs and your budget. This will allow you to deal with the immediate impact of
the event, and then recover your data to new systems from the copies warehoused
off-site after they are returned to you by the contractor.
It is at this level of disaster that you will also have to
deal with non-technical issues well before your technology plant can come back
online. Level 5 disasters almost always include loss of physical space and—unfortunately—loss
of life as well. When your employees are no longer available to enact a DR
plan, you will need to act as quickly as possible, given the situation, to find
new staffers, train them, and get things up and running again. Also keep in
mind the immense psychological impact of these kinds of disasters. Employees
have probably just lost their homes, possibly their family members, and friends
as well. Attempts to coerce such employees to immediately report back to work
is both unfair and in many cases unethical, which could leave some large gaps
in your DR efforts. Temporary staff may be available in some cases for you to
use in the short term, but for the majority of cases you will simply have to redefine
your DR plan to take the extra recovery time into consideration.
The best planning you can do for a Level 5 emergency is to
prepare everyone for what they can expect, and hold firm if executives try to
make you commit to anything unreasonable. Set up phone chains and other
alerting structures ahead of time, get your data out of the scope of potential
disasters that may impact your production environment, and be ready to deal
with the harsh consequences of a massive disaster. The best you can do is to
prepare: Level 5 disasters will find every hole your DR plan has to offer.