On Energy-aware Allocation and Execution for Batch and Interactive MapReduce

The energy-performance optimization of datacenters becomes ever challenging, due to heterogeneous workloads featuring different performance constraints. In addition to conventional web service, MapReduce presents another important workload class, whose per-formance highly depends on data availability/locality and shows different degrees of delay sensitivities, such as batch vs. inter-active MapReduce. However, current energy optimization solu-tions are mainly designed for a subset of these workloads and their key features. Here, we present an energy minimization frame-work, in particular, a concave minimization problem, that specif-ically considers time variability, data locality, and delay sensitivity of web, batch-, and interactive-MapReduce. We aim to maximize the usage of MapReduce servers by using their spare capacity to run non-MapReduce workloads, while controlling the workload de-lays through the execution of MapReduce tasks, in particular batch ones. We develop an optimal algorithm with complexity O(T 2) in case of perfect workload information, T being the length of the time horizon in number of control windows, and derive the struc-ture of optimal policy for the case of uncertain workload informa-tion. Using extensive simulation results, we show that the proposed methodology can efficiently minimize the datacenter energy cost while fulfilling the delay constraints of workloads. 1