A friend of mine is a system administrator for an East Coast company with a secondary (DR) data center at a colocation in a western state. We chatted recently about some of the features of his secondary data center, which is dedicated to maintaining security and uptime for its clients. With his insights fresh in mind (and with his permission), I thought it might be interesting to outline the processes used by the hosting organization — which I'll leave nameless for confidentiality purposes — by discussing what you might expect to encounter were you to visit it.
(Insert shimmering clouds and harp-like sounds of an imagination at work….)
Arriving at the colo
You pull up to the steel gate, which completely encloses the facility from the outside world, and buzz the security guard at the colocation. The facility operates 24x7x365 with security staff on premises at all times — yes, Thanksgiving and Christmas Day too. The guard asks you to identify yourself and you do so into the intercom, showing your company badge to the camera as well. He doesn't ask, but you remove your sunglasses to make it clear your face matches your ID. He checks to confirm your identity is legitimate and that you're on the approved list to enter the colocation. You pass the test, so he opens the gate. You drive in and the gate is immediately closed behind you.
Coming up the hill, you're treated to an impressive view of the building, which resembles a one-story warehouse without any windows except for a handful near the main entrance. That's the only entrance, except for a loading dock, although there are one-way exit-only fire doors that set off an alarm when opened. Situated near the Rockies, this site can withstand earthquakes, floods, tornadoes, blizzards, and other disasters... including physical attack, as demonstrated by the bulletproof glass and Kevlar-lined concrete walls. You notice the absence of telephone and electrical lines running to the building. They're there, but you can't see them since they're underground, and there are several of them to ensure redundancy.
You park out front, get out of the car, and buzz a second intercom to obtain access to the building. The door clicks as the lock is released. The video camera recording your entry through the door is behind steel and unbreakable glass. Get used to being recorded. Cameras abound throughout the facility, even in customer areas. Especially in customer areas, to protect their contents.
You greet the security guard, who is behind more bulletproof glass, and show your ID again. He asks you to sign in. Once you do so, he provides you with an access badge and requests that you wear it at all times. You can now go to your company data center (known as a "cage," since each company has its own private cage to store its servers and other equipment).
You walk down the hall and use your access badge to enter a room that reminds you of an airlock. This is in fact a sort of airlock; not in the literal sense, but it's an anteroom in which the entrance door and the exit doors can't be open at the same time. You pull the door behind you and then use the badge a second time to open the interior door.
Beyond the "airlock"
Now you're in the inner keep. The air temperature and humidity levels in here are strictly controlled by environmental systems to ensure that things remain dry and cool (but not too dry, since static electricity is undesirable). The hallway stretches left and right and you can see dozens of intersecting hallways lined with customer cages. These cages are locked both physically (requiring a key to open) and electronically (requiring badge access). Alarms help further ensure against unauthorized access. The cages are vertical and horizontal, completely enclosing customer assets — no climbing in or underneath. It's possible to see inside every cage, and the variety of their contents is interesting. Some are loaded with servers, while others have just a couple of racks. Some are messy and others are examples of tidy Naval precision. The facility staff makes sure nothing is kept in a customer cage that should not be there — flammable liquids, for instance.
You enter your company's cage after unlocking the door and using your badge, which has been coded to permit access to only your cage. (Biometric access is also available, but your company opted against it.) Your entry into the cage illuminates an indicator at the guard's station, but that's okay since he's aware of your presence and can see you on the monitor. Similarly, opening the cage has sent your IT team back at HQ an email alert, so you notify them via email that "It's just me onsite."
Inside the cage, your servers and network equipment sit humming in a row of locked racks. Everything is redundant: redundant power, cooling, and network services (including the MPLS lines connecting this backup data center with your company's primary data center 1,000 miles away). The colocation facility features smoke detectors, an FM200 gas fire suppression system, and onsite technicians who are there to assist if needed and to make sure things stay safe and predictable. Several generators ensure that the site can keep running indefinitely even if the area loses electricity. Your primary site has a better chance of suffering a long-term outage, which is why your organization selected this colocation for its DR data center.
Getting to work
You're there to replace a network switch, so you locate the FedEx box and open it. You shipped this switch to the colocation facility the previous week and the staff scanned the box in their mailroom to make sure it contained no harmful materials before delivering it to the cage. The facility doesn't monitor your equipment, so there's no need to notify them that you'll be removing the existing switch. But monitoring services of this nature are available for extra cost, along with hands-on requests, such as swapping backup tapes, running network cables, and even installing servers.
Minutes later, your work is done and you take some pictures of the new switch in the rack to add to your network documentation. It's okay to photograph the interior of your cage, but taking pictures of other company cages or the inside of the building is a no-no. The guard will politely request that you delete the images while he watches, should you commit that error.
Time to go
After locking your cage up once more you exit the main floor into the "airlock" room. As before, you let the inner door close before opening the outer one. It's required to sign out from the facility, so you do so and hand over your visitor badge, then bid the guard so long. A tape deliveryman enters with a locked box of tapes for another company; evidently it has requested these from its offsite tape storage company. The guard asks a nearby site technician to accompany the tape man to the appropriate cage — outside visitors must be escorted and watched at all times.
Exiting the gate, you drive out of the facility and note the way you have to make a sharp right to get onto the main road. This is by design so that approaching cars can't ram the gate. In similar fashion, you see a plane off in the distance heading west while you're driving north. The facility has been specifically built away from airline flight paths so that a plane crash poses no risk.
Wrapping up the tour
That's my friend's company colocation in a nutshell. As you can see, there is a rigid set of policies afoot and no concept of "We'll bend the rules just this once." This facility doesn't believe in do-overs or cutting corners. It's protecting billions of dollars in client revenue in a building that would probably make an ideal shelter for the beleaguered characters of AMC's The Walking Dead (except for the fact food is prohibited in the main floor area, so they'd have to scrounge supplies elsewhere). Whether you're considering a colocation facility, looking for ways to secure your onsite data center, or just interested in the Mission Impossible realm of secured data center hosting, I hope this tour has been an interesting ride.
Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.