ACIC: Automatic Cloud I/O Configurator for HPC Applications

The cloud has become a promising alternative to tradi-tional HPC centers or in-house clusters. This new environ-ment highlights the I/O bottleneck problem, typically with top-of-the-line compute instances but sub-par communica-tion and I/O facilities. It has been observed that changing cloud I/O system configurations leads to significant varia-tion in the performance and cost efficiency of I/O intensive HPC applications. However, storage system configuration is tedious and error-prone to do manually, even for experts. This paper proposes ACIC, which takes a given applica-tion running on a given cloud platform, and automatically searches for optimized I/O system configurations. ACIC utilizes machine learning models to perform black-box per-formance/cost predictions. To tackle the high-dimensional parameter exploration space unique to cloud platforms, we enable affordable, reusable, and incremental training guided by Plackett and Burman Matrices. Results with four repre-sentative applications indicate that ACIC consistently iden-tifies near-optimal configurations among a large group of candidate settings