The Design of an Auto-Tuning I/O Framework on Cray XT5 System

Abstract—As high performance computing (HPC) heads towards the exascale era, the computing power surges tremen-dously and applications will scale to hundreds of thousands cores. Consequently, the amount of data processed and gener-ated will increase dramatically. Nowadays, a parallel shared file system is a must have for a supercomputer. To utilize I/O effectively is essential for an application to scale up. We have developed a mathematical model to describe parallel I/O activities that serves as the basis for an I/O auto-tuning infrastructure for HPC systems. Our current work is in the context of Lustre, but our ideas should be applicable to other distributed file systems. This paper explains our model, which is based on queuing theory, describes the auto-tuning process, and gives experimental results over Lustre on the Cray XT5 that show low relative error. Keywords-I/O; performance modeling; auto-tuning; queuing theory; simulation I