Pattern-driven Parallel I/O Tuning

The contemporary parallel I/O software stack is complex due to a large number of configurations for tuning I/O per-formance. Without a proper configuration, I/O becomes a performance bottleneck. As high performance computing (HPC) is moving towards exascale, poor I/O performance has a significant impact on the runtime of large-scale simu-lations producing massive amounts of data. In this paper, we focus on developing a framework for tuning parallel I/O configurations automatically. This auto-tuning framework first traces high-level I/O accesses and analyzes data write patterns. Based on these patterns and historically avail-able tuning parameters for similar patterns, the framework selects best performing configurations at runtime. If pre-vious history for a pattern is unavailable, the framework initiates model-based training to acquire efficient set of tun-ing parameters. Our framework includes a runtime system to apply the selected configurations using dynamic linking, without the need for changing application source code. In this paper, we describe this framework and evaluate it using multiple I/O kernels extracted from real applications and demonstrate substantial I/O performance improvement. 1