Petuum: A New Platform for Distributed Machine Learning on Big Data

Provided by: Association for Computing Machinery
Topic: Big Data
Format: PDF
How can one build a distributed framework that allows efficient deployment of a wide spectrum of modern advanced Machine Learning (ML) programs for industrial-scale problems using big models (100s of billions of parameters) on big data (terabytes or petabytes)? Contemporary parallelization strategies employ ne-grained operations and scheduling beyond the classic bulk-synchronous processing paradigm popularized by Map Reduce, or even specialized operators relying on graphical representations of ML programs. The variety of approaches tends to pull systems and algorithms design in different directions and it remains difficult to find a universal platform applicable to a wide range of different ML programs at scale.

Find By Topic