Data Management

Fusing Data Management Services With File Systems

Date Added: Nov 2009
Format: PDF

File systems are the backbone of large-scale data processing for scientific applications. Motivated by the need to provide an extensible and flexible framework beyond the abstractions provided by API libraries for files to manage and analyze large-scale data, the authors' are developing Damasc, an enhanced file system where rich data management services for scientific computing are provided as a native part of the file system. This paper presents the vision for Damasc, a performant file system that would allow scientists or even casual users to pose declarative queries and updates over views of underlying files that are stored in their native bytestream format.