Join or sign in Register for your free TechRepublic membership or if you are already a member, sign in using your preferred method below. Use Your Email Use FacebookUse Linkedin

Test DR plans on a regular basis

Locked

Does your organization test its disaster recovery plan on a regular basis, or did you only test it in the early stages of implementation? Do you agree with Mike Talon that continued testing is important? Share your comments about testing DR plans on a regular basis, as discussed in the Dec. 16 Disaster Recovery e-newsletter.

If you haven’t subscribed to our free Disaster Recovery e-newsletter, sign up today!
http://nl.com.com/acct_mgmt.jsp?brand=techrepublic

Topic

only way

In reply to Test DR plans on a regular basis

Testing your plan every year (or at least major parts of it) is probably the only way to find the big holes.

In our 2003 test, we went to test one system and found it didn’t work at all. Reason: we didn’t restore the middle tier server!

A little research showed why: the support people just forgot about it, and more importantly, the “dashboard” system that indicates which systems are up/down/slow didn’t even list it!

Furthermore, the service level agreement for that system didn’t list it either!

We also found that a failover server for another system would have failed over to 8-month old data because the dialy script that copies changes from the production system to the failover system hadn’t worked in ages.

Most interestingly, I also noticed that the MS SQL server that one my admins created from scratch for one test had NO patches on it to prevent a Blaster infection (SQL SP3).

So testing separates the sheep from the goats and gives you good insight into your procedures. Everyone hates doing recovery testing, but a lot of CYA work is done quickly after holes are revealed.

correction

In reply to Test DR plans on a regular basis

I meant SQL Slammer, not blaster.

test vs exercise vs practice

In reply to Test DR plans on a regular basis

Semantics, semantics. When we view DR simulations as actual “tests”, we promote the idea that a success negates the need for further testing. This is obviously not true because DR simulations are more than verification of functionality. They are events that provide the opportunity to hone skills, to practice old and new techniques, and to bring new blood into the process of recovering the organization.

Testing indicates success or failure in many corporate cultures. Yet, as BC/DR professionals, I’m sure many of us understand that finding failures in a simulation is the best time to find them. So, a failure in turn becomes a success if you are able to isolate the root cause and implement corrective measures.

Just as an athlete exercises to maintain top performance ability, DR plans need exercising to ensure they can perform to the level of expectations during a time of crisis.

DR simulations are also a very valid conduit for gathering requirements. When something in the organization changes, especially on the business side where the information may not trickle over to the DR group in a timely manner, a DR simulation exposes this new requirement when the business groups make it known that they can’t perform some of their job functions without x, y, or z.

I suppose what I am trying to get across is that as an industry we need to get away from labeling the time spent simulating recovery of systems, applications, processes, and functions as “tests” and promote the idea that these activities are an ongoing part of the larger program that is disaster recovery and continuity management.

Somewhat agree

In reply to test vs exercise vs practice

The words we choose are important, but not only to us as DR/HA professionals. Most corporate culture will demand “testing,” using that word.

Call the exercise whatever you will, management will be looking for the resulst of the “test.”

Final Step to Testing

In reply to Test DR plans on a regular basis

The final step in the DR Plan testing/exercise process is the postmortem meeting and management report. This is often overlooked as an important part of testing. A Management report should document all of the test objectives that were and were not met, as well as the challenges and issues that came up during the test. All of these issues, in turn, should be documented in a project task list to ensure they are addressed in the DR plan in the form of updates and/or changes.
Not only does this report serve to inform management of the state of their DR planning process, but also aids in future audits by internal, external and regulatory auditors by showing what was accomplished, and that changes were made to the DR plan based on the test’s findings. The postmortem meeting is to finalize any thoughts about the test by the participants, which should also aid in the planning of future tests.

[Author]

Replies