A Characteristic Study on Failures of Production Distributed Data-Parallel Programs

Download Now
Provided by: Tsihai
Topic: Big Data
Format: PDF
SCOPE is adopted by thousands of developers from tens of different product teams in Microsoft Bing for daily web-scale data processing, including index building, search ranking, and advertisement display. A SCOPE job is composed of declarative SQL-like queries and imperative C# User-Defined Functions (UDFs), which are executed in pipeline by thousands of machines. There are tens of thousands of SCOPE jobs executed on Microsoft clusters per day, while some of them fail after a long execution time and thus waste tremendous resources. Reducing SCOPE failures would save significant resources.
Download Now

Find By Topic