Data Centers

Improving the Availability of Supercomputer Job Input Data Using Temporal Replication

Date Added: May 2009
Format: PDF

Storage systems in supercomputers are a major reason for service interruptions. RAID solutions alone cannot provide sufficient protection as 1) growing average disk recovery times make RAID groups increasingly vulnerable to disk failures during reconstruction, and 2) RAID does not help with higher-level faults such failed I/O nodes. This paper presents a complementary approach based on the observation that files in the supercomputer scratch space are typically accessed by batch jobs whose execution can be anticipated. Therefore, the authors propose to transparently, selectively, and temporarily replicate "Active" job input data by coordinating the parallel file system with the batch job scheduler.