On Power Consumption Profiles for Data Intensive Workloads in Virtualized Hadoop Clusters

IEEE International Conference on Computer Communications (INFOCOM 2017). 1 to 4, May, 2017, Workshop Big Data and Cloud Performance. Atlanta, U.S.A..Although reduction in operating costs remains to be a key motivation for migration to Cloud environments, Power consumption is a big concern for data centers and cloud service providers. Many big data applications execute on Hadoop MapReduce framework for processing large workloads. In this paper, we investigate the tradeoff between energy consumption and workload running on Hadoop clusters using multiple virtual machines. We characterize power consumption profiles for various data intensive workloads and correlate these to quality of service (QoS) metrics such as job execution time. Based on experiments, we ascertain that power consumption profiles for big data applications can be used to optimize energy efficiency in data centers. We infer that these profiles can be used by Cloud service providers and consumers to specify green metrics in Service Level Agreements (SLA).