Chunk and Object Level Deduplication for Web Optimization: A Hybrid Approach
Proxy caches or Redundancy Elimination (RE) systems have been used to remove redundant bytes in WAN links. However, they come with some inherited deficiencies. Proxy caches provide less savings than RE systems, and RE systems have limitations related to speed, memory and storage overhead. In this paper, the authors advocate the use of a hybrid approach, in which each type of cache acts as a module in a system with shared memory and storage space. A static scheduler precedes the cache modules and determines what types of traffic should be forwarded to which module. They also propose several optimizations for each of the modules, such that the storage and memory overhead are minimized.