A Comparative Experimental Study of Parallel File Systems for Large-Scale Data Processing

Large-scale scientific and business applications require data processing of ever-increasing amounts of data, fueling a demand for scalable parallel le systems comprising hundreds to thousands of disks. Modern parallel le system architectures however, span a large and complex design space. As a result, IT architects are faced with a challenge when deciding on the most appropriate parallel le system for a specific scientific or industrial application in a large-scale computing installation. Typically, the right choice depends on the characteristics of the application as well as the design assumptions built into a parallel le system.