Discussion Locked

Cross training and disaster recovery

0 helpful
6 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.
Tell us what you think about Mike Talon's advice on having more than one person responsible for each DR task, as featured in the August 27th Disaster Recovery e-newsletter. Does your DR plan rely on the actions of a single individual?

6 replies

by chirish

... and involve everyone.

The greatest "single point of failure" in any DR/BC plan is usually the lack of adequate and regular reviews and/or testing.

The best way to ensure contact and personnel lists are maintained is to involve the HR department in your DR/BC processes. Also include the DR/BC responsibilities into job descriptions for each position that is involved in any of your plans, including the resposibility for reviewing the relevant plans, understanding their role(s), and ifrequired, a review/test of the plan(s).

by draco vulgaris

Is it good enough to be executed by random people? I am a VMS System Manager and good at what I do. I would hate to have to take over the job of restoring a Unix server! I have some some Unix training and experience but lack the breadth and depth of experience that would allow me to quickly troubleshoot anything that went wrong. A Unix system administrator would probably have similar problems restoring VMS systems.

Is my plan good enough? I don't know. During our last DR test I spent about half the available time debugging scripts that I couldn't test on production systems. During our next test, I hope to have our help desk specialist try to restore my systems using my plan and scripts. Only then will I know if it's good enough or, if not, have some idea of what's needed to make it good enough.

by ed

The three most important words in the execution of any DR/BC plan. Too many IT shops feel they can handle a DR. However, when it comes time to actually run the test, too many generalists and not enough documentation. The real experienced personnel in DR are you mainframers. Yes, we may be old, but we've done this many times over and over and over again. Patince is the keyword!!! have good documentation and make sure that anyone can bring up your systems in a disaster...You may not be around...The other caution is repetitive tests of your DR. First test, your basic system (infrastructure, Enterprise backup system(s) and build servers in preparation for a restore. Second test, same as first but add at least 1 major application. Third test is a repeat of the previous plus a few more applications. Fourth test is fill-blown. Then you start all over...

by gpsmc

There should always be a secondary person listed in a DR plan for each role / task. Everybody should have a deputy in most roles anyway, if they go on holiday, are off sick etc. Any plan that doesn't have a backup is a badly written plan! As for cross-training - it's highly undesireable to have the receptionist try to bring back up servers if the entire IT team is unavailable and that's taking it too far. But when I'm writing plans I have each recovery task set out in horrendous detail on a stage basis. I'm currently writing a DR plan for a technology dependent company that only has (at a push) half a dozen technical staff - the question of what happens if someone is unavailable is an acute one in that situation.

by editor's response

Here's a comment from one of our readers:

I think Mike has the bull by the horns! I don't know how many times I have run into problems, and not just disasters, where the entire action failed because the only person who knew what was needed was away from the desk, out of the office, or just off BSing somewhere. This situation is running absolutely rampant, and until people will take their respective duties responsibly and train others to act in their absence it will continue to do so. I think many people feel that if they tell someone else how to do their job, it might get done better, or done without them, which makes them very nervous. Likely nervous about the stability of their job or making a decision that is less than the best and getting a severe lack of support from a superior who is probably just as insecure. They may be justified in this feeling, particularly if they are not exactly someone's fair-haired person! This feeling comes mostly from being insecure, and in this day and age, that is justified also, because loyalty and trust have all but disappeared in industry, the government and this country (could be extended to the rest of the world, for that matter). Not much chance of finding a solid answer to this, but Ido wish whoever tackles the problem the best of luck.

by editor's response

Here's another comment from one of our readers:

This is absolutely essential, for any organization with more than a handfull of people. And the importance only increases with the number of staff in a organization - regardless if the organization is a business, emergency service provider, or volenteer group. Disorganization will affect the groups effectiveness and efficiency in various degrees, and in some cases, prohibit the group from functioning.

One broken link in the chain reduces a bike from being a rideable vehicle to a pedestrian burden. Keep your chain clean, oiled, and up to date.

This discussion is closed to new replies.