Data Centers

Facebook open sources new data-center tech at the 2015 OCP summit

Facebook's ground-up data-center redesign is saving the company billions of dollars and energy. Read about Facebook's announcements at the 2015 Open Compute Project Foundation summit.

facebookocpfiga031615.jpg
Figure A: Facebook had approximately 1.3 billion active users as of Q4 2014.
Image: Statistica.com

When people are asked what they like about Facebook, few if any mention the company being an innovative pioneer in data-center engineering. Actually, in a roundabout way, that's a tribute from more than 1.3 billion Facebook users to those who work in Facebook data centers.

Redesign from the ground up

To continue to provide good service to an ever-increasing number of members (Figure A) required Facebook to add data-center infrastructure and equipment designing to its repertoire in 2009 — that's when Facebook management tasked a group of company engineers with designing a data center from the ground up. Two years later, Facebook's data center in Prineville, Oregon went online. Once the data center was dialed-in, operators determined the Prineville facility was using 40% less energy than other Facebook data centers.

The following concepts, software, and equipment developed by Facebook engineers are the reasons the Prineville and the newer Altoona, Iowa data centers are so efficient:

Open Compute Project

Facebook realized the importance of its efforts and made public all pertinent information about the above technology through the Open Compute Project (OCP) Foundation. "We believe that openly sharing ideas, specifications, and other intellectual property is the key to maximizing innovation and reducing operational complexity in the scalable computing space," explains the project's mission statement. "The Open Compute Project Foundation provides a structure in which individuals and organizations can share their intellectual property."

Announcements from the 2015 OCP summit

The OCP Foundation had its 2015 U. S. summit last week and there were a slew of interesting announcements from Frank Frankovsky, president and chairman of the OCP Foundation. "I'm happy to say that the OCP community's influence has gained a lot of momentum in the past year, with new contributions and membership from companies like HP, Dell, Cisco, Apple, and Microsoft," said Frankovsky on the first day of the summit. "We have nearly 200 companies now participating in the project, and every day new technologies are being developed and contributed."

Frankovsky also disclosed that Facebook contributed the specifications for Wedge to the OCP Foundation. The press release mentions, "Facebook is working with Accton, Broadcom, Cumulus, and Big Switch to create a Wedge product package for the OCP community. Accton will begin shipping Wedge in the first half of 2015."

Figure B

facebookocpfigb031615.jpg
A depiction of how all the component technologies come together in the Wedge ToR switch.
Image: Facebook

The social-media giant also released FBOSS Agent, software based on Broadcom's OpenNSL library and created to run on Facebook's Wedge, and OpenBMC, a system-management platform. "Recently, we found the Baseboard Management Controller (BMC) software stack too closed to meet our needs, so we built our own version, which we're open-sourcing today," explains Tian Fang, a software engineer at Facebook. OpenBMC will be first deployed on Wedge and then 6-pack. The diagram in Figure B depicts how all the component technologies come together in the Wedge ToR switch.

Facebook's big announcement

The big announcement from Facebook at the summit was the company's new System On a Chip (SoC) platform code-named Yosemite. "Over the last 18 months, Facebook has been working with Intel on a new SoC compute server that dramatically increases speed and more efficiently serves Facebook traffic," according to Facebook's press release. "Yosemite is our first system-on-a-chip compute server that supports four independent servers (Mono Lake) at a performance-per-watt superior to traditional data-center servers for heavily parallelizable workloads."

Hu Li, Facebook hardware-design engineer, introduced Yosemite (Figure C) in this post, mentioning that it includes the following design elements:

  • Server-class SoC with multiple memory channels, which provides high-performance computing in 65W TDP for SoC and 90W for the whole server card.
  • Standard SoC card interface to provide a CPU-agnostic system interface.
  • Platform-agnostic system management solution to manage the system and these 4 SoC server cards, regardless of vendor.
  • Multi-host network interconnect card following OCP Mezzanine Card 2.0 specification, which connects up to 4 SoC server cards through a single Ethernet port.
  • Cost-effective, flexible, and easy-to-service system structure.

Figure C

facebookocpfigc031615.jpg
Facebook's new System On a Chip (SoC) platform is code-named Yosemite.
Image: Facebook

Facebook isn't finished

At the summit, Facebook representatives disclosed that improvements engineers made to the company's data-center infrastructure has saved Facebook $2 billion in infrastructure costs over the past three years. "We're not finished — not even close," hinted that more was to come.

Also read

About Michael Kassner

Information is my field...Writing is my passion...Coupling the two is my mission.

Editor's Picks

Free Newsletters, In your Inbox