Portals and Networking for the Lustre File System

Date Added: Jan 2010
Format: PDF

Sandia/University of New Mexico Operating System (SUNMOS) and the Puma were the first type of operating systems used. These were based on light weight kernels and were used for the large scale production platforms with thousands of processors. Over the past few years, the need for improving the technology of portals was felt. Therefore, the portals technology, which was based on proprietary vendor parallel platforms, was changed to large scale commodity-based PC clusters. The interface and semantics were re-designed so that they can be used to support scalable, high-performance data movement using intelligent network interfaces such as Myrinet. An important characteristic of portals is that they provide low-level, flexible building blocks that can be combined to support higher-level protocols efficiently. In addition, portals support the functional requirements of high-performance file systems like Lustre very well. This paper discusses how this re-defined technology in portals has helped in providing an optimal foundation for networking the lustre file system. Lustre is a scalable parallel file system designed to be used for large scale computer clusters. Lustre needs to support many different networks including Ethernet, InfiniBand, Myrinet and Quadrics. With Sandia National Laboratories developing API, it was showed that portals message passing provides the finest base for networking of the Lustre file system. Remote DMA and request processing are some of the storage networking features of this portal.