pNFS, POSIX, and MPI-IO: A Tale of Three Semantics

Date Added: Nov 2009
Format: PDF

Advanced research collaborations push the bounds of modern technology, but continue to be constrained by rigid computing and storage infrastructures. Many large compute clusters are tightly coupled with a single file system, requiring large data sets to be moved multiple times across a computational grid. MPI-IO is emerging as the standard mechanism for file I/O within HPC applications. While pNFS demonstrates high-performance I/O for bulk data transfers, its performance and scalability with MPI-IO is unproven. To attain success, the consistency semantics and interfaces of pNFS, POSIX, and MPI-IO must all be reconciled and efficiently translated. This paper investigates and discusses the challenges of using pNFS to support the consistency semantics of HPC applications.