Data Management

XORing Elephants: Novel Erasure Codes for Big Data

Free registration required

Executive Summary

Distributed storage systems for large clusters typically use replication to provide reliability. Recently, erasure codes have been used to reduce the large storage overhead of three-replicated systems. Reed-Solomon codes are the standard design choice and their high repair cost is often considered an unavoidable price to pay for high storage efficiency and high reliability. This paper shows how to overcome this limitation. The authors present a novel family of erasure codes that are efficiently repairable and offer higher reliability compared to Reed-Solomon codes. They show analytically that their codes are optimal on a recently identified tradeoff between locality and minimum distance.

  • Format: PDF
  • Size: 1781.76 KB