A power-measurement methodology for large-scale, high-performance computing

Improvement in the energy efficiency of supercomputers can be accelerated by improving the quality and comparability of efficiency measurements. The ability to generate accu-rate measurements at extreme scale are just now emerging. The realization of system-level measurement capabilities can be accelerated with a commonly adopted and high quality measurement methodology for use while running a workload, typically a benchmark. This paper describes a methodology that has been developed collaboratively through the Energy Efficient HPC Work-ing Group to support architectural analysis and compara-tive measurements for rankings, such as the Top500 and Green500. To support measurements with varying amounts of effort and equipment required we present three distinct levels of measurement, which provide increasing levels of ac-curacy. Level 1 is similar to the Green500 run rules today, a single average power measurement extrapolated from a subset of a machine. Level 2 is more comprehensive, but still widely achievable. Level 3 is the most rigorous of the three methodoloiges but is only possible at a few sites. How-ever, the Level 3 methodology generates a high quality re-sult that exposes details that the other methodologies may miss. In addition, we present case studies from the Leibniz Supercomputing Centre (LRZ), Argonne National Labora-tory (ANL) and Calcul Québec Universite ́ Laval that ex-plore the benefits and difficulties of gathering high quality, system-level measurements on large-scale machines