Automatic and Transparent I/O Optimization With Storage Integrated Application Runtime Support

Traditionally storage has not been part of a programming model’s semantics and is added only as an I/O library in-terface. As a result, programming models, languages, and storage systems are limited in the optimizations they can perform for I/O operations, as the semantics of the I/O li-brary is typically at the level of transfers of blocks of uninter-preted bits, with no accompanying knowledge of how those bits are used by the application. For many HPC applica-tions where I/O operations for analyzing and checkpointing large data sets are a non-negligible portion of the overall ex-ecution time, such a “know nothing ” I/O design has negative performance implications. We propose an alternative design where the I/O seman-tics are integrated as part of the programming model, and a common data model is used throughout the entire memory and storage hierarchy enabling storage and application level co-optimizations. We demonstrate these ideas through the integration of storage services within the Legion [2] runtime and present preliminary results demonstrating the integra-tion. 1