Questions

Server room power backup

Tags:
+
0 Votes
Locked

Server room power backup

I've been tasked with creating a backup power plan for our server room. We have three tower servers and really only one of them is mission critical. We also have the router, firewall, and phone system which will need to remain on during a power outage. We need at least the one server to be able to stay up through any outage...our most recent power outage was about three hours but I suppose we should be prepared for anything. Any recommended hardware as far as battery backup, generator, etc?
  • +
    0 Votes
    brian

    yeah, a big generator would be recommended for that application (something that sits outside and runs off diesel like you see hidden in many large city parks)...by using a generator in theory you could run the critical systems indefinatly (as long as you have a fuel supply).

    +
    0 Votes
    robo_dev

    At a high level, you need to determine exactly how much power your devices require and how much runtime you need. And you need to figure out your budget.

    Off the top of my head, when you say 'any outage', that takes the price tag from several thousand to many tens of thousands.

    For a couple of thousand, a nice rack-mount 3KVA UPS for your server would probably run it for a couple of hours, and another 3KVA unit would run most phone systems for an hour or so.

    APC makes a product line called Symmetra that is scalable, you just add more battery modules.

    http://www.apc.com/tools/ups_selector/index.cfm

    Some phone systems, especially legacy ones, need a lot more power than 3KVA. However most legacy phone systems already have a UPS built into the cabinet.

    If you're talking about 'any outage', then you would need a generator with fuel tank (or natural gas supply), an automatic transfer switch, a bank of batteries, and the related chargers, etc. etc. Easily well over $30-40K.

    +
    0 Votes
    OH Smeg

    May be a starting point for you

    http://www.apc.com/sizing/selectors.cfm

    Click on UPS enter your Country and then fill in the forms and you have a basic starting point.

    But the reality is first you need to know how long you are planning for a Power Outage and how much power that you need for what has to be run. Most long term Mains Replacement involves Generators or Solar Arrays depending on your location on the globe.

    Col

    +
    0 Votes
    TobiF

    Define a diesel, which will be able to keep everything running (with margins, of course.)

    Next figure out, what is the longest time needed to start the diesel. This time you need to cover with UPS. (again, with good margins, obviously, since systems tend to grow over time)

    If your system is really important, don't forget about redundancy. Either 2 units, each of which can cover all the need, or 3 servers, for half the total capacity each.

    For servers, which are not mission critical, it may be enough to have UPS cover enough time for an emergency back-up of live data and/or controlled shut-down.

    +
    0 Votes
    jck

    If you need a solution to maintain uptime no matter what, your solution is going to have to be a power generation system.

    You'd be able to get away with something pretty basic. You don't have tons of stuff to keep online. You will just need to work with some firm to make sure you have an auto-start system in place that will kick in the generator in the event of an outage, and provide power to the facility for a period of time after the power returns from the grid.

    It's not a cheap solution (compared to battery backup). But for mission-critical systems that could be without mains power for 1 hour or more, backup generation is the best option to ensure you can stay up for longer periods of time.

    +
    0 Votes
    robo_dev

    You still need a UPS of sufficient size to carry the load until the generator can start and get up to speed.

    A generator allows you runtime that is only limited by your access to fuel, but it's not an either-or solution.

    If you've got a generator, you may be able to save a couple of dollars on the UPS, since your power requirement will be for much less runtime. However, you still need enough UPS to carry the load and provide power to the automatic transfer switch.

    +
    0 Votes

    Well I've decided to get the quotes for a generator that can power the entire server room, but not the rest of the building. The electrician was here yesterday scouting out what we would need to do to hook it all up. I'm thinking that a ten minute battery back up should be sufficient with the generator. Does that sound good?

    +
    0 Votes
    TobiF

    Even if your office is manned 24/7, it would be good to have a little bit more margin, in case you need to manually restart the generator (or refuel it).

    +
    0 Votes
    robo_dev

    :)

    +
    0 Votes
    OH Smeg

    I remember the very Famous Failure of the Backup Generator during the Moon Landing where it had been serviced but the Injector Pump not Primed so that when the power failed the Generator didn't have any fuel in the Injectors to start.

    Could be in the same situation if it's never serviced the fuel goes stale and is no longer capable of running the engine.

    Col

    +
    0 Votes
    robo_dev

    While most of the big generators are diesel, natural gas is a convenient alternative for the smaller generators.

    Obviously natural gas does not go stale, and you do not need a storage tank, nor does a truck need to deliver fuel.

    In the US there are a multitude of environmental regulations related to fuel storage tanks as well as capturing fuel spills, etc.

    In high-end systems you install a 'fuel polishing' system that circulates/cleans/filters the diesel fuel, much like a swimming pool filtration system. Algae growth is a real problem in diesel storage tanks.

    +
    0 Votes
    fatman65535

    Some suggestions from the trenches:

    First, you need to understand the sequence of events that are involved with generators and power failures.

    1) Utility power lost.
    2) A time out period allowed for utility power to restore before starting the generator. (You do not want to start up a generator for a 10 second outage.) If you can NOT tolerate ANY power loss (mission critical systems) then a UPS system will have to provide power during this interval.
    3) Generator start up time, and stabilization. UPS needs to supply power during this time also.
    4) Automatic (preferable) or manual transfer of load from (dead) utility service to generator. UPS still needs to supply power until generator is actually connected.
    5) Monitor utility power for restoration, with a restoration time delay. What this time delay does is prevent restoration of power from generator to utility when the utility is experiencing interruption issues. (Thunderstorms causing instantaneous power failures, and restorations aka `blinking lights`.)
    6) Transfer of load from generator to the restored utility, the UPS needs to supply some power during that interval.

    Specifically for # 4 and 6. Most transfer switches are designed so that one side BREAKS before the other side makes. This is done so the generator and utility power do not get ACCIDENTALLY CONNECTED TO EACH OTHER (aka BACKFEED). The switch interval can be only a second or two long, but in that interval, no power, the UPS has to power mission critical equipment.

    Maintenance Issues

    1) Battery status, BOTH UPS and GENERATOR START BATTERIES. UPSes are useless if their batteries are DEAD. Ditto with generators. Check them periodically. UPS batteries need to be able to provide their capacity for the entire expected power loss duration.
    2) You must run the generator periodically to insure that it will be ready when you need it. That usually means a minimum of 30 minutes per month, possibly more depending on the generator set manufacturers recommendations. This is no different that the need to start and run a car that is in storage.
    3) On a regular basis, perform periodic oil changes. Ideally, the company that installs your generator will perform a maintenance check at least once a year. This is one form of our sourcing I do recommend. (Since I live in hurricane country, we schedule our yearly maintenance in May, before the season starts.)
    4) Also, you should schedule as part of the yearly maintenance, a SIMULATED POWER FAILURE to insure that the AUTOMATIC startup and load transfer occurs as you would expect IT TO.
    5) Have enough fuel, make sure the fuel tank is full. Nothing is more aggravating than to THINK you have a 2500 gallon tank full of fuel, only to find out it is less than half full. (Remember when gas and diesel got real expensive??? Some people decided to `help themselves`.)
    6) Make sure that your UPS has the capacity to run through the steps to transfer power, anticipate failure to make the power transfer, and still have sufficient power to allow you enough time to do an ORDERLY shutdown. Remember, if it CAN fail, then it WILL FAIL. Cover Your A--.


    When my employer found out that in my earlier days, I WAS ONCE an electrician, guess WHO got put in charge of making sure the generator worked? I encountered some **** back from the lower ranked bean counters in Finance, but all it took was a 10 minute discussion with the CEO (read as OWNER) of the company, and those bean counters were told to SHUT UP!

    +
    0 Votes
    Kenone

    an Eaton UPS that gives me a theoretical 11 hours of runtime (13 servers most virtual) and an orderly shut down after the batteries run to 15%. So far the longest outage I've weathered has been about 6 hours, no problems.

  • +
    0 Votes
    brian

    yeah, a big generator would be recommended for that application (something that sits outside and runs off diesel like you see hidden in many large city parks)...by using a generator in theory you could run the critical systems indefinatly (as long as you have a fuel supply).

    +
    0 Votes
    robo_dev

    At a high level, you need to determine exactly how much power your devices require and how much runtime you need. And you need to figure out your budget.

    Off the top of my head, when you say 'any outage', that takes the price tag from several thousand to many tens of thousands.

    For a couple of thousand, a nice rack-mount 3KVA UPS for your server would probably run it for a couple of hours, and another 3KVA unit would run most phone systems for an hour or so.

    APC makes a product line called Symmetra that is scalable, you just add more battery modules.

    http://www.apc.com/tools/ups_selector/index.cfm

    Some phone systems, especially legacy ones, need a lot more power than 3KVA. However most legacy phone systems already have a UPS built into the cabinet.

    If you're talking about 'any outage', then you would need a generator with fuel tank (or natural gas supply), an automatic transfer switch, a bank of batteries, and the related chargers, etc. etc. Easily well over $30-40K.

    +
    0 Votes
    OH Smeg

    May be a starting point for you

    http://www.apc.com/sizing/selectors.cfm

    Click on UPS enter your Country and then fill in the forms and you have a basic starting point.

    But the reality is first you need to know how long you are planning for a Power Outage and how much power that you need for what has to be run. Most long term Mains Replacement involves Generators or Solar Arrays depending on your location on the globe.

    Col

    +
    0 Votes
    TobiF

    Define a diesel, which will be able to keep everything running (with margins, of course.)

    Next figure out, what is the longest time needed to start the diesel. This time you need to cover with UPS. (again, with good margins, obviously, since systems tend to grow over time)

    If your system is really important, don't forget about redundancy. Either 2 units, each of which can cover all the need, or 3 servers, for half the total capacity each.

    For servers, which are not mission critical, it may be enough to have UPS cover enough time for an emergency back-up of live data and/or controlled shut-down.

    +
    0 Votes
    jck

    If you need a solution to maintain uptime no matter what, your solution is going to have to be a power generation system.

    You'd be able to get away with something pretty basic. You don't have tons of stuff to keep online. You will just need to work with some firm to make sure you have an auto-start system in place that will kick in the generator in the event of an outage, and provide power to the facility for a period of time after the power returns from the grid.

    It's not a cheap solution (compared to battery backup). But for mission-critical systems that could be without mains power for 1 hour or more, backup generation is the best option to ensure you can stay up for longer periods of time.

    +
    0 Votes
    robo_dev

    You still need a UPS of sufficient size to carry the load until the generator can start and get up to speed.

    A generator allows you runtime that is only limited by your access to fuel, but it's not an either-or solution.

    If you've got a generator, you may be able to save a couple of dollars on the UPS, since your power requirement will be for much less runtime. However, you still need enough UPS to carry the load and provide power to the automatic transfer switch.

    +
    0 Votes

    Well I've decided to get the quotes for a generator that can power the entire server room, but not the rest of the building. The electrician was here yesterday scouting out what we would need to do to hook it all up. I'm thinking that a ten minute battery back up should be sufficient with the generator. Does that sound good?

    +
    0 Votes
    TobiF

    Even if your office is manned 24/7, it would be good to have a little bit more margin, in case you need to manually restart the generator (or refuel it).

    +
    0 Votes
    robo_dev

    :)

    +
    0 Votes
    OH Smeg

    I remember the very Famous Failure of the Backup Generator during the Moon Landing where it had been serviced but the Injector Pump not Primed so that when the power failed the Generator didn't have any fuel in the Injectors to start.

    Could be in the same situation if it's never serviced the fuel goes stale and is no longer capable of running the engine.

    Col

    +
    0 Votes
    robo_dev

    While most of the big generators are diesel, natural gas is a convenient alternative for the smaller generators.

    Obviously natural gas does not go stale, and you do not need a storage tank, nor does a truck need to deliver fuel.

    In the US there are a multitude of environmental regulations related to fuel storage tanks as well as capturing fuel spills, etc.

    In high-end systems you install a 'fuel polishing' system that circulates/cleans/filters the diesel fuel, much like a swimming pool filtration system. Algae growth is a real problem in diesel storage tanks.

    +
    0 Votes
    fatman65535

    Some suggestions from the trenches:

    First, you need to understand the sequence of events that are involved with generators and power failures.

    1) Utility power lost.
    2) A time out period allowed for utility power to restore before starting the generator. (You do not want to start up a generator for a 10 second outage.) If you can NOT tolerate ANY power loss (mission critical systems) then a UPS system will have to provide power during this interval.
    3) Generator start up time, and stabilization. UPS needs to supply power during this time also.
    4) Automatic (preferable) or manual transfer of load from (dead) utility service to generator. UPS still needs to supply power until generator is actually connected.
    5) Monitor utility power for restoration, with a restoration time delay. What this time delay does is prevent restoration of power from generator to utility when the utility is experiencing interruption issues. (Thunderstorms causing instantaneous power failures, and restorations aka `blinking lights`.)
    6) Transfer of load from generator to the restored utility, the UPS needs to supply some power during that interval.

    Specifically for # 4 and 6. Most transfer switches are designed so that one side BREAKS before the other side makes. This is done so the generator and utility power do not get ACCIDENTALLY CONNECTED TO EACH OTHER (aka BACKFEED). The switch interval can be only a second or two long, but in that interval, no power, the UPS has to power mission critical equipment.

    Maintenance Issues

    1) Battery status, BOTH UPS and GENERATOR START BATTERIES. UPSes are useless if their batteries are DEAD. Ditto with generators. Check them periodically. UPS batteries need to be able to provide their capacity for the entire expected power loss duration.
    2) You must run the generator periodically to insure that it will be ready when you need it. That usually means a minimum of 30 minutes per month, possibly more depending on the generator set manufacturers recommendations. This is no different that the need to start and run a car that is in storage.
    3) On a regular basis, perform periodic oil changes. Ideally, the company that installs your generator will perform a maintenance check at least once a year. This is one form of our sourcing I do recommend. (Since I live in hurricane country, we schedule our yearly maintenance in May, before the season starts.)
    4) Also, you should schedule as part of the yearly maintenance, a SIMULATED POWER FAILURE to insure that the AUTOMATIC startup and load transfer occurs as you would expect IT TO.
    5) Have enough fuel, make sure the fuel tank is full. Nothing is more aggravating than to THINK you have a 2500 gallon tank full of fuel, only to find out it is less than half full. (Remember when gas and diesel got real expensive??? Some people decided to `help themselves`.)
    6) Make sure that your UPS has the capacity to run through the steps to transfer power, anticipate failure to make the power transfer, and still have sufficient power to allow you enough time to do an ORDERLY shutdown. Remember, if it CAN fail, then it WILL FAIL. Cover Your A--.


    When my employer found out that in my earlier days, I WAS ONCE an electrician, guess WHO got put in charge of making sure the generator worked? I encountered some **** back from the lower ranked bean counters in Finance, but all it took was a 10 minute discussion with the CEO (read as OWNER) of the company, and those bean counters were told to SHUT UP!

    +
    0 Votes
    Kenone

    an Eaton UPS that gives me a theoretical 11 hours of runtime (13 servers most virtual) and an orderly shut down after the batteries run to 15%. So far the longest outage I've weathered has been about 6 hours, no problems.