Applying machine learning towards automating resource management in cloud computing environments

Over the last number of years there has been high acceleration and widespread adoption of cloud based services by many different industries to improve the scalability and reliability of their services while also reducing costs. The popularity of the cloud has invoked major concern for the high energy consumption and carbon emissions evident in the operation of data centers today. One of the key causes of high energy consumption is inefficient resource management. While much effort in recent years has been devoted to achieving improved energy efficient resource management strategies, performance uncertainty has also become a major hindrance for cloud resource management systems. Performance uncertainties present significant challenges for planning and provisioning Virtual Machine (VM) resource allocation and scheduling in the cloud while having negative impacts overall in achieving greater energy efficiencies and improved Quality of Service (QoS). Unlike traditional resource management strategies that rely on heuristics this work presents novel forecasting based approaches for scheduling and allocating the computational resources of public cloud infrastructure. Our goal is to extend and make advancements upon existing research by applying and comparing the performance of state of the art learning algorithms using real world data. We explore how these advanced approaches can be incorporated to give rise to intelligent resource management systems with the capacity to make improved decisions under uncertainty. In particular, we propose and analyse several interesting variations and extensions to problems such as workflow scheduling, dynamic VM consolidation and also VM migration. The results obtained from this research indicate the immense potential of adopting learning based methodologies to optimise resource usage, improve performance and overall attempt to reach new frontiers in energy efficiency