Facebook, Google and Microsoft each rely on a global network of datacenters – vast warehouses stacked with tens of thousands of servers and hundreds of miles of cable.
These highly automated, maximum security facilities are generally kept away from the prying eyes of the public, but every so often the tech giants lift the lid on what goes on inside.
Here’s a look inside some of the datacenters used by the big three, offering a peek at the sheer scale of their infrastructure.
This is Facebook’s main Arctic datacenter in Lule\u00e5, Sweden, a facility measuring 300m long by 100m wide — or about the size of 7.5 football fields.
Facebook was drawn to town by it’s low winter temperatures, about -30C (-22F), and plentiful supply of power from nearby hydro-electric generators.
The datacenter, which was Facebook’s first European facility when it opened in 2013, is so large that engineers use scooters to get around.
When running so many machines, simplicity is a must, and in 2014 the datacenter became the first to use the Open Compute Project’s ‘rapid deployment’ design, allowing trucks to carry in pre-assembled server chassis.
These empty halls are the norm, as only about 150 people work at Lule\u00e5. Due to the heavily automated nature of the datacenter, Facebook only needs one technician for every 25,000 servers.
The drive for a rapid turnaround also extends to repairs, with Facebook reducing the time it takes to replace a server hard drive from an hour to two minutes.
Equipment is also stripped back to the basics in order to keep it as cold as possible, necessary as the center relies on fresh-air cooling and summer temperatures can still reach 20C.
These fans pull in cold air to chill the servers and during the winter can push out the heat generated by the servers to keep other buildings in the facility warm.
These mangled disks are all that remains of the old drives used to store Facebook data. Facebook CEO Mark Zuckerberg says the company ensures disks are “crunched, forever protecting privacy”. However, it’s worth noting the debate over how Facebook handles user data extends to far more than destruction of old equipment.
Facebook server technician Christer Jonsson holds the remains of old drives.
Local craftsman and artists contributed to the look of the exterior of the building.
A Facebook engineer stands in front of some of the many miles of network cabling that runs throughout the datacenter.
Lule\u00e5’s climate has its disadvantages. “The biggest challenge working here? Getting to the datacenter by car when it’s -30C outside,” says technician Emilie De Clercq.
As well as cars, staff use skis and snowmobiles to travel to work over the frozen ground.
Google runs some one million servers, spread over 15 datacenters across the globe.
The scale of each facility is captured by this shot of the floor of its facility in Council Bluffs, Iowa. The massive steel beams both support the structure and carry the power cables that run throughout the datacenter.
The Iowa datacenter occupies more than 115,000 square feet and employs over 300 people.
The racks of servers seen here are just a single server cluster on one floor in the multi-storey datacenter.
Like Facebook, managing this many machines requires heavy automation, with Google using its own Borg, Colossus and Spanner software to take care of many server-related tasks.
Much of the infrastructure is also custom-made, which Google says provides the flexibility and performance needed to run its services at such a large scale.
The reassuring blue glow given off by LEDs on the servers is a good sign, as it means the machines are running smoothly.
The data stored by Google is routinely backed up.
Seen here is one of the tape libraries Google uses in its South Carolina datacenter, where barcoded tapes are stored and retrieved using robot arms.
This eyecatching cooling plant, decked out in Google’s colors, channels water around the facility to control the temperature of the 70,000+ heat-spewing servers on each floor.
A similarly snazzy color scheme is also applied to cables, not for show but to make it easier for engineers to immediately tell different leads apart.
Behind the server aisle, hundreds of fans funnel hot air from the computers into a cooling unit, so it can be recirculated.
Relative to most other datacenters, Google’s facilities are highly efficient in the energy they expend on cooling, with an average PUE of 1.12 across its sites.
Inside the facility’s network room, routers and switches allow Google’s data centers to talk to each other.
Google’s datacenters are connected by private links, which it says run faster than its internet-facing network and help its cloud platform services access infrastructure in any location.
Huge amounts of floor space are required to house the number of servers Google needs.
In Hamina, Finland, the firm converted an old paper mill, which as well as its size had the added advantage of being close to the Gulf of Finland’s cooling waters.
Each server rack has four switches, connected by a different colored cable.
The same color scheme is used throughout the datacenter, to make it easy to identify which cable needs replacing.
Google’s data center in The Dalles, Oregon sits on the banks of the Columbia River and is powered by nearby hydro-electric power plants.
A single building in the Google Cloud Platform datacenter can support 75,000 machines and carry over one petabit per second of bandwidth.
This is Microsoft’s datacenter complex in Quincy, Washington.
Microsoft opened its first datacenter in the city in 2007.
The Columbia 1 and 2 datacenters on the Quincy site occupy the equivalent of one dozen US football fields and house three generation of datacenter designs.
These server racks are part of Microsoft’s generation 5 design, which Microsoft describes as ‘software-defined’ and energy efficient, with potential for a PUE as low as 1.07.
Microsoft has been constantly refining the design of its datacenters, recently introducing modular datacenters, or datacenter as a box, which can be rapidly deployed and offers better energy efficiency than earlier designs.
These generation 4 servers don’t rely on huge uninterruptible power supplies for backup, instead having batteries fitted to each server.
More modular datacenter enclosures at Quincy, this time with a roof.
The modular, ‘generation 4’ datacenter design uses adiabatic cooling, which Microsoft plans to use in its future datacenter designs.
In addition to this existing facility, Microsoft is building its new ‘generation 5’ datacenter on the 270-acre site in Quincy.
Hydropower power plants on the Columbia River produce electricity for Quincy.
Microsoft has set a target for half of the power it uses to be produced by wind, solar or hydropower by 2018.
Nick Heath is a computer science student and was formerly a journalist at TechRepublic and ZDNet.
