Storage

Comanche - A Compiler-Driven I/O Management System

Date Added: Jan 2011
Format: PDF

Most scientific programs have large input and output data sets that require out-of-core programming or use Virtual Memory Management (VMM). Out-of-core programming is very error-prone and tedious; as a result, it is generally avoided. However, in many instance, VMM is not an effective approach because it often results in substantial performance reduction. In contrast, compiler driven I/O management will allow a program's data sets to be retrieved in parts, called blocks or tiles. Comanche (COmpiler MANaged caCHE) is a compiler combined with a user level runtime system that can be used to replace standard VMM for out-of-core programs. The authors describe Comanche and demonstrate on a number of representative problems that it substantially out-performs VMM.