EndRE: An End-System Redundancy Elimination Service for Enterprises

In many enterprises today, WAN optimizers are being deployed in order to eliminate redundancy in network traffic and reduce WAN access costs. In this paper, the authors present the design and implementation of EndRE, an alternate approach where Redundancy Elimination (RE) is provided as an end system service. Unlike middleboxes, such an approach benefits both end-to-end encrypted traffic as well as traffic on last-hop wireless links to mobile devices. EndRE needs to be fast, adaptive and parsimonious in memory usage in order to opportunistically leverage resources on end hosts. Thus, they design a new fingerprinting scheme called SampleByte that is much faster than Rabin fingerprinting while delivering similar compression gains.