Panda: Fast access to persistent arrays using high-level interfaces and server directed input/output

Multidimensional arrays are a fundamental data type in scientific computing and are used extensively across a broad range of applications. Often these arrays are persistent, i.e., they outlive the invocation of the program that created them. Portability and performance with respect to input and output (i/o) pose significant challenges to applications accessing large persistent arrays, especially in distributed-memory environments. A significant number of scientific applications perform conceptually simple array i/o operations, such as reading or writing a subarray, an entire array, or a list of arrays. However, the algorithms to perform these operations efficiently on a given platform may be complex and non-portable, and may require costly customizations to operating system software.This thesis presents a high-level interface for array i/o and three implementation architectures, embodied in the Panda (Persistence AND Arrays) array i/o library. The high-level interface contributes to application portability, by encapsulating unnecessary details and being easy to use. Performance results using Panda demonstrate that an i/o system can provide application programs with a high-level, portable, easy-to-use interface for array i/o without sacrificing performance or requiring custom system software; in fact, combining all these benefits may only be possible through a high-level interface due to the great freedom and flexibility a high-level interface provides for the underlying implementation.The Panda server-directed i/o architecture is a prime example of an efficient implementation of collective array i/o for closely synchronized applications in distributed-memory single-program multiple-data (SPMD) environments. A high-level interface is instrumental to the good performance of server-directed i/o, since it provides a global view of an upcoming collective i/o operation that Panda uses to plan sequential reads and writes. Performance results show that with server-directed i/o, Panda achieves throughputs close to the maximum AIX file system throughput on the i/o nodes of the IBM SP2 when reading and writing large multidimensional arrays.U of I OnlyETDs are only available to UIUC Users without author permissio