When a company activates its disaster recovery plan, the IT department’s overall role is to reconstruct the IT infrastructure and restore systems and services to their predisaster state. At first blush, the help desk team’s role in the process may be limited or even on hold until the recovery is complete.
However, the help desk can and should play a pivotal role in the company’s recovery efforts. This week, I’d like to propose a few items that belong on the help desk’s disaster recovery preparedness checklist and invite you to add your suggestions to the list.
The phases of a typical recovery
For the purpose of this discussion, let's assume the disaster is a flood that has made it impossible for anyone to get into the corporate office and the data center for an indefinite amount of time. A typical IT department’s overall disaster recovery plan looks something like this:
Step 1: First response
Senior management declares the disaster and activates the plan, mobilizing the recovery teams.
Step 2: Infrastructure restoration
Designated members of the IT recovery team arrive at the hot site. This could be space rented from a provider of disaster recovery services or the IT manager’s basement. The team assesses available resources and starts rebuilding the network, establishing remote-access entry points and e-mail services so employees can get in to the network once it is sufficiently restored.
Step 3: Data restoration
Once the infrastructure is rebuilt, the IT team begins restoring server-based production data and applications from the most recent backups.
Step 4: Resynchronization of data
During recovery of the IT infrastructure and network databases, the folks on the business-unit side will have been busy using manual processes for tasks such as gathering and processing payroll data, paying vendors, and fulfilling customer orders. Depending on how long the network has been down, there may be a considerable backlog of information to be synchronized with the data restored from backups.
Step 5: Steady-state operations
If the corporate office is going to be inaccessible for an extended period of time, employees should conduct business by using remote-access connections to get to their applications. This situation isn’t ideal, but it’s better than trying to operate the business via manual processes.
Step 6: Return to normal operations
During this step, the business units and IT team begin the process of migrating back to the home office.
Step 7: Evaluate the results of the recovery plan
Once business operations have returned to a more or less normal state, the plans need to be evaluated by everyone involved in the recovery effort. The disaster recovery plans should be updated accordingly.
The help desk’s role
One of the key elements of any disaster recovery plan is the telephone tree, a list of names and numbers used to contact employees. When the plan is activated, the first people called are the team leaders, who in turn call the people in their business units. Some employees will be called and told to stay at home, while others will be asked to report to the hot site.
Employees are accustomed to contacting the help desk with tech support questions. So the help desk team is an excellent choice to make outgoing telephone-tree calls and respond to incoming questions about availability of the system. A representative from the help desk team should be intimately involved with restoring the infrastructure and steady-state operations, as well as with resynchronizing the data.
The help desk team should, of course, have its own disaster recovery plan, and the team members should maintain their own off-site copies of that plan. Here are some of the items that should be included in the plan and stored at the hot site or in a safe place away from the office:
- A master contact list—You’ll need a list of all help desk team members and their home telephone numbers, pager numbers, or home e-mail addresses to stay in touch with each other.
- A copy of the standard desktop image—A CD with the most recent standard image for corporate desktop machines will come in handy if you have to build a desktop from scratch for employees who will be working from the hot site or from home.
- Installation media for core business applications—These backup copies should include your company’s e-mail client and any productivity tools. You’ll want them handy if you need to get an application that isn’t on the standard image up and running in a hurry.
- A database for tracking issues during the recovery process—You don’t stop creating trouble tickets just because you’re in recovery mode. When employees in the field attempt to connect to the network and run applications at the various stages of recovery, you need a way to track those users and their issues.
The specific procedures and tools needed for responding to a disaster will vary from company to company. But in every organization, the help desk is in a position to play a key role in restoring normal business operations.
What’s your recovery plan?
How has your technical support staff prepared for disaster recovery? To share your tips for recovering normal operations after a disaster, please post a comment or write to Jeff.