Supporting Computational Data Model Representation with High-Performance I/O in Parallel NetCDF
Parallel computational scientific applications have been described by their computation and communication patterns. From a storage and I/O perspective, these applications can also be grouped into separate data models based on the way data is organized and accessed during simulation, analysis, and visualization. Parallel netCDF is a popular library used in many scientific applications to store scientific datasets and provides high-performance parallel I/O. Although the metadata-rich netCDF file format can effectively store and describe regular multi-dimensional array datasets, it does not address the full range of current and future computational science data models. In this paper, the authors present a new storage scheme in Parallel netCDF to represent a broad variety of data models used in modern computational scientific applications.