Automatic Identification of Application I/O Signatures from Noisy Server-Side Traces

Competing workloads on a shared storage system cause I/O resource contention and application performance va-garies. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help al-leviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive appli-cations. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are al-ready collected on today’s supercomputers, without in-terfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbu-lence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our ex-periments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accu-racy and shorter processing time.