Measuring and tuning energy efficiency on large scale high performance computing platforms

Recognition of the importance of power in the field of High Performance Computing, whether it be as an obstacle, expense or design consideration, has never been greater and more pervasive. Research has been conducted in a number of areas related to power. Little, if any, existing research has focused on large scale High Performance Computing. Part of the reason is the lack of measurement capability currently available on small or large platforms. Typically, research is conducted using coarse methods of measurement such as inserting a power meter between the power source and the platform, or fine grained measurements using custom instrumented boards (with obvious limitations in scale). To collect the measurements necessary to analyze real scientific computing applications at large scale, an in-situ measurement capability must exist on a large scale capability class platform. In response to this challenge, the unique power measurement capabilities of the Cray XT architecture were exploited to gain an understanding of power use and the effects of tuning both CPU and network bandwidth. Modifications were made at the operating system level to deterministically halt cores when idle. Additionally, capabilities to alter operating P-state were added. At the application level, an understanding of the power requirements of a range of important DOE/NNSA production scientific computing applications running at large scale (thousands of nodes) is gained, by simultaneously collecting current and voltage measurements on the hosting nodes. The effects of both CPU and network bandwidth tuning are examined and energy savings opportunities of up to 39% with little or no impact on run-time performance is demonstrated. Capturing scale effects was key. This thesis provides strong evidence that next generation large-scale platforms should not only approach CPU frequency scaling differently, but could also benefit from the capability to tune other platform components, such as the network, to achieve energy efficient performance.National Nuclear Security Agency (NNSA) Advanced Simulation and Computing (ASC) program and the Department of Energy’s (DOE) Innovative and Novel Computational Impact on Theory and Experiment (INSITE) program.Electrical and Computer EngineeringMastersUniversity of New Mexico. Dept. of Electrical and Computer EngineeringShu, WeiShu, WeiPollard, HowardAng, Jame