Scalable I/O Forwarding Framework for Petascale Architectures

Abstract—Current leadership-class machines suffer from a significant imbalance between their increasing computational power and the limited I/O bandwidth. While Moore’s law ensures that the computational power of high-performance computing (HPC) systems increases with every generation, the same is not true for their I/O subsystems. The limited scalability of existing parallel file systems, coupled with the minimalistic compute node kernels running on these machines, calls for a new I/O paradigm to meet the requirements of data-intensive scientific applications. I/O forwarding is a technique that attempts to bridge the increasing performance gap between the compute and I/O components of HPC systems by shipping I/O calls from compute nodes to dedicated I/O nodes. The I/O nodes perform I/O on behalf of the compute nodes and can reduce file system traffic by aggregating, rescheduling, and caching I/O system calls. This paper presents an open, scalable I/O forwarding framework capable of running on massively parallel HPC systems such as the IBM BG/P, Cray XT5, and Linux clusters. We also describe an I/O protocol and API for shipping function calls from compute nodes to I/O nodes, and we present a quantitative analysis of the overhead associated with I/O forwarding