Energy Proportionality and Workload Consolidation for Latency-critical Applications

Energy proportionality and workload consolidation are im-portant objectives towards increasing efficiency in large-scale datacenters. Our work focuses on achieving these goals in the presence of applications with µs-scale tail latency re-quirements. Such applications represent a growing subset of datacenter workloads and are typically deployed on dedi-cated servers, which is the simplest way to ensure low tail latency across all loads. Unfortunately, it also leads to low energy efficiency and low resource utilization during the fre-quent periods of medium or low load. We present the OS mechanisms and dynamic control needed to adjust core allocation and voltage/frequency set-tings based on the measured delays for latency-critical work-loads. This allows for energy proportionality and frees the maximum amount of resources per server for other back-ground applications, while respecting service-level objec-tives. Monitoring hardware queue depths allows us to detect increases in queuing latencies. Carefully coordinated adjust-ments to the NIC’s packet redirection table enable us to re-assign flow groups between the threads of a latency-critical application in milliseconds without dropping or reordering packets. We compare the efficiency of our solution to the Pareto-optimal frontier of 224 distinct static configurations. Dynamic resource control saves 44%–54 % of processor en-ergy, which corresponds to 85%–93 % of the Pareto-optimal upper bound. Dynamic resource control also allows back-ground jobs to run at 32%–46 % of their standalone through-put, which corresponds to 82%–92 % of the Pareto bound