Scalable Techniques for Performance Analysis

The scalability of performance tools in high performance computing has been lagging behind the growth in the sizes of supercomputers and the applications that run on them. It is recognized that performance event traces still provide the best and most useful source of information for analysis. However, the volume of performance trace data generated easily becomes unmanagable without appropriate controls as we scale upwards. At the same time, the amount of information that has to be presented to a human analyst can also become overwhelming. We present techniques used to address the above problems and enhance the scalability of Projections, a performance instrumentation and visualization framework for the migratable object programming model CHARM++. Projections provides multiple resolutions of performance data. We couple this feature with the use of heuristics and clustering algorithms at application runtime to provide powerful mechanisms to reduce performance data volume and analysis time while preserving data relevance. We employ similar heuristics and algorithms for enhanced interactive post-mortem visualization and analysis assistance to a human expert. To demonstrate the continued effectiveness of the analysis process using the above techniques while showing that the data volume is significantly reduced, we present experimental results on simulation benchmarks based on NAMD, a popular molecular dynamics application.