Innovation

Data scientists can use MLPerf to see how fast their machine learning tools truly are

Created by Google, Intel, Baidu, and others, the new benchmarking tool helps users understand and improve performance of machine learning hardware and software.

Building a slide deck, pitch, or presentation? Here are the big takeaways:
  • A group of tech industry and academic leaders released MLPerf, a benchmark for measuring the speed of machine learning software and hardware.
  • MLPerf is meant to accelerate improvements in machine learning system performance.

On Wednesday, a group of tech industry and academic leaders released a new benchmark to measure the speed of machine learning software and hardware and accelerate improvements in system performance.

The benchmark—called MLPerf—was created by a number of tech companies including AMD, Baidu, Google, and Intel, as well as researchers from educational institutions including Harvard, Stanford, and the University of California Berkeley.

MLPerf measures speed based on the time it takes to train deep neural networks to perform tasks such as recognizing objects, translating languages, and playing the game of Go.

SEE: IT leader's guide to the future of artificial intelligence (Tech Pro Research)

As machine learning and artificial intelligence (AI) efforts grow across the enterprise, systems need to evolve quickly to meet demands, Andrew Ng, founder and CEO of Landing.AI, said in a press release. "AI is transforming multiple industries, but for it to reach its full potential, we still need faster hardware and software," Ng said in the release.

MLPerf's goal is to speed improvements in machine learning system performance the same way that the Standard Performance Evaluation Corporation (SPEC) benchmark did for general purpose computing. After the SPEC's release in 1988, CPU performance improved 1.6x per year for the next 15 years, the release noted.

"MLPerf combines best practices from previous benchmarks including: SPEC's use of a suite of programs, SORT's use one division to enable comparisons and another division to foster innovative ideas, DeepBench's coverage of software deployed in production, and DAWNBench's time-to-accuracy metric," the release stated. "Benchmarks like SPEC and MLPerf catalyze technological improvement by aligning research and development efforts and guiding investment decisions."

The team is developing MLPerf using the agile framework, in that the benchmark will launch early, involve a large community, and iterate rapidly, the release noted. To learn more, you can visit mlperf.org for a complete specification with reference code, and future results. Hardware vendors and software framework providers can submit their results using the benchmark by July 31.

Also see

istock-932559358.jpg
Image: iStockphoto/monsitj

About Alison DeNisco Rayome

Alison DeNisco Rayome is a Senior Editor for TechRepublic. She covers CXO, cybersecurity, and the convergence of tech and the workplace.

Editor's Picks

Free Newsletters, In your Inbox