General discussion

Locked

Local Volcano maybe awaking

By tundraroamer ·
A "local" volcano may erupt in the near future causing some grief. While not in any danger from the actual eruption, the ash fallout will be the problem. It can produce prolonged power outages and the lots of grit in the air. I am working on contingency plans and can cover the basics. What I need to know is what are items or events that may be less obvious that I need to plan for that may not be limited to just IT problems and protection of equipment. In particular, what about offsite operation to run the business? What are the experiences from those that have gone through other types of natural disasters?

This conversation is currently closed to new comments.

29 total posts (Page 1 of 3)   01 | 02 | 03   Next
| Thread display: Collapse - | Expand +

All Comments

Collapse -

You need a complete disaster-recovery plan

by DC_GUY In reply to Local Volcano maybe awaki ...

To fully prepare for a disaster of this magnitude, you must have a full disaster-recovery plan. You can't possibly be ready for a volcano eruption that close to your data center without it. Depending on what business you're in and how much down time it can withstand, you may need a "cold" site or even a "hot" site. You'll certainly need off-site backups.

Sitting there in your comfortable chair on a nice day, you can probably do a halfway decent job of imagining what will need to be done if the volcano erupts. But when the eruption happens and everyone is thinking about their homes and families instead of their jobs, no one -- including yourself -- will be able to think clearly.

You have to have a detailed plan to follow, and you may even need to hold practice drills.

I've been through a few earthquakes, so I know what I'm talking about.

Collapse -

Practical

by Black Panther In reply to You need a complete disas ...

I have to do DR for the Gov, including disasters, acts of God etc etc. One thing management and auditor's seem to forget is that Computers run on electricity - if you don't have a generator then no matter how good your DR is - if you have no electricity you are stuffed.

Also they forget about the Customer's, Suppliers etc - if they have no DR - their will be no deliveries etc etc

Sometimes you need to think laterally and practically.

Collapse -

FUEL FOR GENERATORS

by Gary.Crispens In reply to Practical

One thing to also keep in mind is that generators need PLENTY fuel to run. Most large generators get terrible fuel mileage in that they might run only 4 hours on a full tank of fuel. This means shutting everything down to refuel the generator.

You can get generators that run on diesel and keep a bunch of 55 gallon drums of fuel on hand. Also remember to get a hand pump for transferring the fuel into the generator.

When Hurricane Isabel hit last fall, many generators ran out of fuel after a day or so and the power was off for over a week in some places. Gas stations need power to pump and many of these were not open for several days up to a week.

Collapse -

Fuel suppliers

by warpindy In reply to FUEL FOR GENERATORS

Gentlemen don't forget to have two fuel suppliers on contract so that if one company can not get you fuel. The other company can still deliver. The storage of fuel drums is also not a bad idea but check with you local and state govement for proper storage and safety guidelines.

Collapse -

Volcano times 2

by tundraroamer In reply to You need a complete disas ...

This is not my first volcano experience if it happens so I kinda know what to expect from this particular volcano. And earthquakes are a way of life here. However, our network and branch locations have grown larger and we are more dependent on our computers then the last time 12 years ago.

I have thought about power issues, people, transportation issues, backups, air filters for both people and equipment but its what I haven't thought of that bothers me. So that's what I am looking for. People that have gone through a natural disaster and later found out what they should have done prior to the event. I think that's the part of the disaster plan I am missing.

Collapse -

Fresh water?

by dazau In reply to Volcano times 2

I recently visited a volcano in Vanuatu and was amazed at the amount of ash that was raining down from a volcano 6 kms away.

You may need fresh water if the catchment is polluted by large amounts of falling ash. Are you using any form of water cooling? Does your generator have a water cooled radiator that may become blocked with ash? Broom/brush etc to keep them clean.

You may also need showering facilities for your key people if they don't want to be all dirty. Some people (esp the opposite sex) can't handle grit and dirt in every orifice and you may need to rely on these people. Keep em happy if you want them to work while everyone else goes home to help their families.

If you have volcanoes then you may have earthquakes/siesmic activity. Do you have backup comms links?

If the ash gets really heavy (was like light rain where I was) then any form of radio or wireless comms may be affected too.

If the ash gets really heavy then it may get dark - need torches/floodlights etc.

And the obvious question - do you have accurate warning system and information when/if the volocano blows. The web is full of monitoring stations and most active volcanoes have monitoring equipment too. Search Google for "volcano monitoring" if you dont already know a good site.

PS: If your trees end up covered in ash then just give them a swift kick around the base! (Works in Vanuatu!)

Collapse -

AGREE TOTALLY

by Gary.Crispens In reply to Fresh water?

Fresh water is always a huge issue after a disaster. We had to either drink bottled water or boil your water for several weeks after Hurricane Isabel hit. The rivers flooded and over ran the sewage treatment center and polluted the groundwater. Trying to buy bottled water in big quantities is impossible after a disaster. Get a dispensor and have plenty of those big commercial bottles of water on hand.

Collapse -

Just a couple of Thoughts...

by rmrwork In reply to Volcano times 2

Geez... a volcano. Yikes.

First off, if you don't have a pre-defined communication plan in place, you should do that ASAP. Be sure to define a procedure for both internal communications and external communications (depending on what sort of company you work for, you may specific obligations that need to be met by you or someone in IT for external communication to the media, government, etc).
Relative to Internal communication, you need to define who in your organization has the specific authority to implement the execution of the D-R plan and begin the recovery / contingency plan that was built. In the event of a disaster, it is key that employees and contractors affiliated with your company know who will be giving the "go" command.
If you do have an existing plan, take a "walk-through" to verify all contact information is current / accurate.

Once the decision to "go" has been given, there should be a clear path of communication to ensure that the plan is executed as planned.
This includes communication with your end-user / client community that relies on your systems. They, especially, need to know who to talk to and who they should be taking their instructions from.

Also, you should include a cascading notification process to notify / activate your key IT or end-user personnel. As noted by one of the other correspondents, they may very well be pre-occupied with the well-being of their own family / property, so a simple paging vehicle probably isn't sufficient: you want a positive response.

You mentioned that your network & branch locations have grown larger and are more dependent on your computers than the last time you went thru this sort of event. Given the current state of end-user technology, if you haven't already done so, someone in IT should conduct a site visit to ensure that their equipment has a corresponding level of care and preparedness to match your systems. Afterall, it will all be for naught if your systems / servers are up and running and their routers and desktops are without power or are damaged.
If it's been 12 years, then there is a good chance that there are differeing levels of expertise managing those end-user devices, some of whom has given little / no thought to backup or recovery. Also, you don't want to be "surprised" by some new desktop / peripheral eqipment that they bought and installed on their own.

Be sure to review all supply chain requirements for your buildings. As some correspondents have noted, be sure that you what supplies that you can or can't expect to get. That includes fuel and power; janitorial service; food / catering; and water for both machines and people.
Once you have ascertained the status of each it will go a long way to validating your plan, or identifying areas of improvement. (For example, if you plan to have people working extended shifts or 24 x whatever, then you'll need food onsite or they will need to leave and then you lose control over them for random periods of time. So you either need a "feeding plan" or a plan that allows them out of the building for extended periods of time.)

Best of luck!

Collapse -

Know where you are in the chain of events

by david In reply to You need a complete disas ...

Generally in DRP and BCP, we set plans that are so subjective we lose sight of the fact that there are threats "upstream" and resultant impacts "downstream".

So when the volcano blows, you may be a thousand miles away, but due to your place in someone else's chain [as well as your own] you are at risk.

eg. the volcano blows ash across a major wine growing district, completely destroying all production - how many jobs are lost downstream? how many businesses fail? what export markets are lost to you [and the wine producers] forever?

So don't just consider personal safety or the loss of infrastructure nearest to the volcano. Think of the ripple effect - from your supplier's suppliers, thru your supplier to you and on to your customers, and their customers.

Create, test and review a plan that looks up and down the chain. And respect those that you depend on and vice versa - it will be the best investment you could make.

That volcano that blew half a world away doesn't look so remote after all - does it?

Collapse -

Disasters are all the same

by zthr2000 In reply to Local Volcano maybe awaki ...

floodin, quakes, wars etc... whan it came to basic survival plannin. Just, still lot's of companies ignore the 3rd and important factor, besides the IT and business survival. And that's an employee's (family and property) survival. With all the respect, what the shi* happens, that's the first thing on everyone's mind. If everyone at your place know that their families and/or property is fine, during this volcano crisis, they'll be ready to focus on tose actual recovery tasks, especialy if (like in your case) disaster is something which could last weeks or months, with good chances of problem's revival. In a such type of possible medium or longer-term crisis (with a repeatin problems, read as DR stages) employees tend to crack, not couse problems at their workplace but at their homes.

Back to IT Employment Forum
29 total posts (Page 1 of 3)   01 | 02 | 03   Next

Related Discussions

Related Forums