Multiprocessor File System Interfaces

MIMD multiprocessors are increasingly used for production super-computing. Supercomputer applications often have tremendous file I/O requirements. Although newer I/O sub-systems, which attach multiple disks to the multiprocessor, permit parallel file access, file system software often has insufficient support for parallel access to the parallel disks, which is necessary for scalable performance. Most existing multiprocessor file systems are based on the conventional file system interface (which has operations like open, close, read, write, and seek). Although this provides the familiar file abstraction, it is difficult to use for parallel access to a file. Scalable applications must cooperate to read or write a file in parallel. We propose an extension to the conventional interface, that supports the most common parallel access patterns, hides the details of the underlying parallel disk structure, and is implementable on both uniprocessors and multiprocessors, It also supports the conventional interface for programs ported from other systems, programmers who do not require the expressive power of the extended interface, and access via a standard network file system. We concentrate on scientific workloads, which on uni processors have large, sequentially accessed files. Parallel file systems and the applications that use them are not sufficiently mature for us to know what access patterns might be typical, but we expect to still see sequential access either locally, within the access pattern of each process, or globally, in the combined accesses of all processes cooperating to access a file