Efficient Resource Utilization for Parallel I /O in Cluster Environments

114 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 1999.In this thesis work, performance factors for parallel I/O on clusters are examined and several algorithms are designed to support parallel I/O efficiently for scientific applications running on commodity clusters, making better utilization of system resources. Specifically, our algorithms are designed to (i) minimize data transfer over the network during I/O if network bandwidth is limited, (ii) reduce message passing latency during I/O of finely-distributed data, (iii) place I/O servers on the appropriate processors in heterogeneous environments, and (iv) balance I/O workload dynamically when necessary. These algorithms have been implemented in the Panda parallel I/O library and tested on several actual and simulated cluster environments. Performance results show that our algorithms improve overall parallel I/O performance significantly with only an insignificant amount of overhead. These algorithms can also be used in other parallel I/O runtime libraries or job schedulers for cluster systems.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD