Download now Free registration required
With a goal of supporting the timely and cost-effective analysis of Terabyte datasets on commodity components, the authors present and evaluate StoreTorrent, a simple distributed filesystem with integrated fault tolerance for efficient handling of small data records. The contributions include an application-OS pipelining technique and metadata structure to increase small write and read performance by a factor of 1-10, and the use of peer-to-peer communication of replica-location indexes to avoid transferring data during parallel analysis even in a degraded state.
- Format: PDF
- Size: 1013.56 KB