A Framework for Analyzing Linux System Overheads on HPC Applications

Linux currently plays an important role in high-end computing systems, but re-cent work has shown that Linux-related processing costs and variablity in network processing times can limit the scalability of HPC applications. Measuring and un-derstanding these overheads is thus key for future use of Linux in large scale HPC systems. Unfortunately, currently available performance monitoring systems introduce large overheads, performance data is generally not available on-line or from the operat-ing system, and the data collected by such systems is generally coarse-grained. In this paper, we present a low-overhead framework for solving one of these problems: mak-ing useful operating system performance data available to the application at runtime. Specifically, we have enhanced Linux Trace Toolkit(LTT) to monitor the performance characteristics of individual system calls and to make per-request performance data available to the application. We demonstrate the ability of this framework to monitor individual network and disk requests, and show that the overhead of our per-request performance monitoring framework is minimal. We also present preliminary measure-ments of Linux system call overhead on a simple HPC.