Power-Aware Value-Based Resource Management in High Performance Computing Systems

In the age of data, there is an increasing demand for higher compute capacity machines. This demand has accelerated the development effort in delivering the first exascale computing system by 2021-23. In the exascale era, power efficiency and system productivity are identified as two of the top ten challenges. With current computing technology, an exascale system is estimated to consume more than 40-50MW. According to the Department of Energy guidelines, an exascale system should consume no more than 20-30 MW of power to keep its operational and installation cost manageable. Therefore an exascale computing system is expected to operate under a system-wide power constraint, and resource manager will treat power as a constraint resource besides computing nodes. The varying constraints on the resources and the increasing computation demand will make a system oversubscribed during its operation. This creates a necessity of a power-aware job scheduler to maximize the system productivity for an oversubscribed and power-constrained computing system. In this work, we utilize a monotonically decreasing time-dependent value function, called job-value, for each job to represent the importance of completing that job for an organization. We quantify the productivity of a high performance computing (HPC) system by accumulating the job-values achieved by the completed jobs in a given time frame. We begin by exposing the trade-offs between two traditionally used static power allocation policies, i.e., uniform and greedy allocation, under different system-wide power constraints. We experimentally demonstrate as the system-wide power budget gets low, the productivity of the greedy allocation can be 38\% higher than the uniform allocation, whereas as the system-wide power budget is relaxed, the uniform allocation can gain up to 6\% higher productivity than the greedy allocation. Inspired by this observation, we then propose a new hybrid power allocation algorithm that adapts its behavior to deliver the combined benefits of the two static allocation strategies. Using an emulation framework of 64-node HPC prototype on a real system, we demonstrate that our adaptive strategy results in improving resource utilization and delivering the productivity that is comparable to the best performing traditional power allocation strategy across different system-wide power constraints. In this study, we also create a simulation framework, to simulate a large-scale HPC system and successfully replicate our findings from our emulation study. We then design an algorithm that dynamically reallocates power from the running jobs to the newly arrived jobs to maximize node usage in the system. Using our simulation framework, we demonstrate that the dynamic power allocation policy improves node utilization and job completion rates by up to 20\% and 9\%, respectively, compared to the traditional static allocation strategies. We also show that the dynamic strategy earns up to 8\% higher productivity, compared to the best performing traditional static allocation strategy under different power constraints. In this work, we also develop two methodologies to create power-execution time models for a class of applications using two different power management techniques. One of the methods creates application-specific models that are parameterized on the CPU power consumption, operating frequency, and idle injections. The second method creates models that are parameterized on the CPU power constraint, node count, core count, and application inputs. We use these models in our static and dynamic power-aware scheduling algorithms to estimate execution time under given resource constraints to make informed scheduling decisions