Measuring High-Performance Computing with Real Applications

The authors discuss the important questions that benchmarking must answer and the degree to which such answers can be given by existing kernel versus real application benchmarks. They describe the state of the art and challenges that must be met to base needed performance measurements on real applications. Finally, they quantify their claims by measuring and comparing several real applications and kernel benchmarks. An important finding is that all three measured computer platforms performed both the best and the worst across the selected applications; best performance significantly depended on the problem being solved, questioning the value of computer rankings that use simplistic metrics