Energy Efficient High-Performance Computing Aware Proactive Dynamic Virtual Machine Consolidation Technique in Cloud Computing

High-Performance Computing (HPC) applications have gained extensive interest in cloud computing. Cloud computing provides HPC applications with enormous on-demand resources; therefore, many studies have tried to investigate the currently available cloud service architectures for running HPC applications in the most effective approach. There is yet a promising research direction to explore techniques to efficiently schedule a mix of HPC and non-HPC applications to increase resource utilization in the cloud data centre. However, there are notable challenges that involve cross-application interference, energy issues, and maintaining service-level agreements (SLAs) in multiple metrics, e.g. response time (web application) vs execution time (HPC application). In High-performance Computing (HPC) applications, the resource usage pattern varies over time, making it challenging to track resource utilization. Better workload predictions will lower energy costs by accurately forecasting future workload demands. For this purpose, proactive dynamic Virtual Machine (VM) consolidation is among the most effective approaches to minimize energy consumption and enhance resource usage while balancing the efficiency of HPC and non-HPC applications in a cloud data centre. Since the VM consolidation problem is strictly NP-hard, many heuristic algorithms have been proposed to tackle the problem. However, these algorithms do not consider the workloads' different nature, incredibly individual classes of HPC workload in the cloud data centre. Thus, this thesis explores the opportunities and challenges of proactive, dynamic VM consolidation for HPC and non-HPC applications on the cloud datacenter. Additionally, this thesis proposes and implements an HPC-aware and energy-efficient scheduling algorithm for proactive, dynamic VM consolidation, which achieves better resource utilization and limits cross-application interference through careful co-location of HPC and Non-HPC applications on the same pool of resources in a cloud datacenter. Experiments were conducted using CloudSim and real HPC workload traces of Metacentrum HPC Workload. The proposed approach, Energy-Aware Multi-Dimensional Online Bin Packing (EAMDOBP), was tested against Power-aware best fit decreasing algorithm (PABFD), Modified Worst Fit decreasing algorithm (MWFD) and Hybrid Local Regression Host Overload Detection (HLRHOD). Results indicate a relative improvement of 60% in Simulation time and a 73% increase in throughput. Additionally, the utilization of the CPU, RAM, and bandwidth increased, respectively, by 77%, 84%, and 70%. The results reveal that the proposed approach significantly improves all the critical metrics used over other heuristic algorithms