Improving I/O Forwarding Throughput with Data Compression
While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, the authors investigate improvements to I/O performance by exploiting this gap. They harness idle CPU resources to compress network traffic, reducing the amount of data transferred over the network and increasing effective network bandwidth. They created a set of compression services within the I/O Forwarding Scalability Layer. These services transparently compress and decompress data as it is transferred over the network. They studied the effect of the compression services on a variety of data sets and conducted experiments on a high-performance computing cluster.