Servers

Performance and Fault Tolerance in the StoreTorrent Parallel Filesystem

Free registration required

Executive Summary

With a goal of supporting the timely and cost-effective analysis of Terabyte datasets on commodity components, the authors present and evaluate StoreTorrent, a simple distributed filesystem with integrated fault tolerance for efficient handling of small data records. The contributions include an application-OS pipelining technique and metadata structure to increase small write and read performance by a factor of 1-10, and the use of peer-to-peer communication of replica-location indexes to avoid transferring data during parallel analysis even in a degraded state.

  • Format: PDF
  • Size: 1013.56 KB