It’s inevitable. At some point in your IT career, you’ll be doing your job, minding your own business, and it will hit you: You’re running out of space in your server room. Or maybe your company or group will move to a new location. In any case, you may be called on to design a new data center for your environment. While this is definitely a lot of work and requires careful planning, the end result can be a center that is built for speed, is secure, and that can grow to accommodate future needs.
In this Daily Drill Down, I’ll provide some tips that will aid you in designing your server room, the hub of your IT infrastructure, so that it will not only fit your current operational needs, but it will also be expandable, secure, and built for speed.
1. The basics
You must consider some basic design decisions before you begin construction of a new data center. First is physical location and room size. When possible, keeping your data away from an exterior wall will generally provide for more security. When it comes to size, plan for growth. The chances that you’ll need to grow your infrastructure are pretty good, and you’ll probably want to keep all of your equipment in one location.
Hardware capacity in a server room is usually the prime factor when determining space. Space is expensive and you want to get the best possible return on your investment. Many organizations use telco racks or enclosed cabinets to make the most of the space they have. Today, with 1U high servers and new blade servers, you can pack anywhere from 42 to hundreds of servers in a single rack.
Measuring the Us
1U is approximately two inches, and 2U is approximately four inches.
Planning for space under a raised floor and a ceiling from 12- to 18-feet high will go a long way toward creating a server room that can be properly cooled and maintained. A higher ceiling allows for more tolerance in the event that the room begins to overheat. Having space under the floor to run cables is critical for rooms of almost any size. It also allows you to use power whips from a central electrical panel rather than having to use a fixed outlet. I much prefer to use power whips since I can run them wherever I want under the floor and can easily add a custom whip of a different rating for equipment with special needs.
Power whips are armored wire cables that are factory-installed so an electrician can connect multiple fixtures to one centralized junction box. This keeps the electrician from having to bring electrical service to each fixture.
For data security, keeping up with service packs, patches, and locking down systems only goes so far. Every security plan also needs to address the physical security of the hardware housing the data and of the infrastructure passing the data. Keeping up with every patch from every vendor will do nothing if someone can walk into your server room and mess with your hardware or walk out of the building with it.
First on the list for security is a strict policy only allowing people who need to be in the room to enter. Using a system such as a keycard with time restrictions is another way to keep track of who is entering and leaving the room and when. Other, more advanced, systems will make your environment even more secure. For example, you could install a handprint recognition system or a system that weighs people on their way into and out of the room to make sure that equipment is not being removed without authorization. It all depends on how much security you require and how much you are willing to spend on it.
Looking for a handprint recognition system? Take a look at the Biometric Consortium for more details.
Another aspect of your security plan is the installation of devices to protect your investment, such as fire detection and suppression systems and water detection systems, depending on where your server room is situated. At one point in my career, I found out how important a water detection system can be. On the floor above the central network hub that served ten buildings, a water problem resulted in gallons of water being filtered by a device that was not designed for water filtering—my network backbone switch. While a water sensor wouldn’t have prevented the actual water problem, it could have allowed me to move the switch before more damage was done.
If you’re looking for a water sensor, take a look at these Google results and take your pick.
When thinking about physical security for your new data center, also consider other environmental factors. Some of these will only appear on your to-do list if you live in certain areas of the country:
- Severe weather—tornados, hurricanes, or frequent, severe lightning storms
- Dry conditions, which increase fire potential
Keeping your server room within environmental limits set by the various manufacturers is absolutely critical to maintaining a reliable, highly available infrastructure. Plan how you will maintain a reasonable temperature and humidity level and supply the appropriate power to the room. As I stated, I like to have the power connections run under the floor on flexible cables. This allows me to place them wherever I need them and to install special power connections, such as a 230-watt whip, if needed. Consider a UPS that can carry the load of all of the critical components of the room for an extended period of time. If you’re running a mission-critical environment, consider a generator that can back up the UPS in the event of a prolonged outage. In addition, procuring power supplies from more than one provider is an important step in ensuring the full-time operation of your environment.
Often an overlooked component to maintaining a reliable infrastructure, monitoring’s importance cannot be overemphasized, especially if you maintain a 24/7 environment. From the beginning, you should plan on setting up systems to monitor every possible aspect of your environment, from servers to network components to monitoring your network traffic. Most monitoring software will allow for notifications to pagers, etc., and I recommend that you take advantage of them. If you’d like to find a low-cost monitoring solution that will cover much more than the basics, take a look at Big Brother and MRTG.
Big Brother and MRTG
If you want information on Big Brother and MRTG, see the following articles: ”Big Brother is watching your network” and ”Enhance MRTG’s network graphing with the Round Robin Database Tool.”
In addition, you should evaluate software packages that can help you protect your data against outside attacks and that verify the integrity of your hardware security solutions, such as firewalls.
5. Cabling for speed and flexibility
One major consideration in the design of a server room is the speed of the connections that it will support. My recommendation: Make it as fast as you can afford. I have seen organizations “low ball” the cabling in a server room only to regret it very soon thereafter. In addition, make sure that all cabling is up to fire codes. It may be a little more expensive initially, but it’s much less expensive than having to replace the cabling and, of course, you can’t compare the cost of the cable to the cost of a life that may be saved by using the proper cabling.
Inside the server room, you should run nothing less than category 6 cabling if you’re building it from scratch. I recommend having the cabling professionally installed by an organization that will certify and give a warranty for its work. Many cabling plants can be guaranteed for five or ten years, potentially saving you a significant amount of downtime and money in the event of a problem with the cable plant. During this process, make sure that your cabling plant is capable of gigabit speeds.
If you have room in the budget, you should also seriously consider fiber runs to strategic points in your server room. The capacity of fiber is much greater than that of copper, and installing it during an initial cabling project is generally less expensive than installing it piecemeal after the fact.
For facilities that include remote cabling closets, run both single and multimode fiber to the remote locations, and a few copper runs to provide the maximum flexibility. You never know what you’ll have to support down the line.
When considering speed, it’s also important to consider how fast new hardware can be rolled out. As I discussed before, a rack-based solution can ease this burden significantly, but it also has other benefits.
Consider a situation in which you may want to reorganize your server room. In a recent server room design, I ran 24 pairs of plenum category 6 cables under a raised floor back to a central network switch from an enclosed cabinet filled with servers. When this cabinet had to be moved to a new location, it was fairly simple. I disconnected the D connector on the end of the cable pair, as well as the power connections to the rack (of which there were six), moved the cabinet, and reconnected the cables. Job done.
Where possible, cable pathways should be provided under the floor to keep it from becoming a tangled rat’s nest, which is all too common in many environments. A rat’s nest can present problems during troubleshooting when a cable needs to be traced through the mess. This can result in extra downtime and has the potential to cost your organization more money.
6. Keep the big picture in mind
During the design phase, consider the effect that each of your design decisions will have on your business and determine whether a particular investment is worth the cost. For example, is uptime critical enough to your business operations to warrant the cost of a backup generator to provide power? When considering the costs of downtime, remember that many factors must be considered, such as staff costs to recover from the outage, employee downtime, lost sales, and poor publicity.
The day after you’re finished building your server room, complete with sufficient power whips under 12 inches of raised floor, 18-foot ceilings, adequate heating and cooling and humidity control, and a complete monitoring solution, you’ll be asked to add something new. If you build your server room properly, this should be a painless process. If you use rack-based servers and get the cabling piece done ahead of time, adding a server to the rack should be a piece of cake and leave you only with the task of adding a new server to the monitoring system.
With some careful planning and a forward look at both technology and your enterprise, your new server room can be built so that it’s secure, flexible, and can allow everything to be done quickly—from transferring data at multigigabit speeds to setting up a new server in minutes.