Data Management

Update Maps - A New Abstraction for High-Throughput Batch Processing

Free registration required

Executive Summary

Key/value databases are popular abstractions for applications that require synchronous single-key look-ups. However, such databases invariably have a random I/O access pattern, which is inefficient on traditional storage media. To maximize throughput, an alternative is to rely on asynchronous batch processing of requests. As applications evolve, changing requirements with regard to scale or load may thus lead to a redesign to increase the use of batch processing. The authors present a new abstraction that they have found useful in making such transitions: the update map. It aims to combine the convenience of a key/value database with the performance of a batch-oriented approach. The interface resembles that of an ordinary key/value database, but its implementation can rely on batch processing and sequential I/O, for improved throughput.

  • Format: PDF
  • Size: 197.52 KB