Intelligent Scaling of Container-based Web Applications in Geographically Distributed Clouds

© 2021 Yasser AldwyanCloud data centers are increasingly distributed around the globe. Recently, containerisation, a lightweight virtualization technology, has been rapidly adopted as an application packaging mechanism for efficient, consistent web application deployment and scaling within and across Cloud-based data centers. To leverage Cloud elasticity and scalability, containers commonly run on elastic, scalable clusters of virtual machines (VMs). Such global infrastructure and lightweight deployment capabilities offer a perfect choice for deploying latency-sensitive web applications in multiple locations to serve globally distributed users. However, managing container-based web applications, including containers and VMs, in widely dispersed data centers currently lacks intelligent deployment and elasticity capabilities from Cloud providers. This thesis investigates several problems related to the lack of such capabilities. This includes problems of deployment such as where and how to deploy VM clusters as well as geo-replicated application containers across data centers to address potential outages while considering wide-area network latency issues. It also considers how to dynamically deploy clusters across data centers to handle potential spatial workload fluctuations with minimum costs. This in turn gives rise to elasticity problems for multi-cluster container-based web applications deployed to multiple data centers. These problems include how to rapidly scale overloaded clusters at the VM level through temporary inter-cluster resource utilisation to avoid Cloud VM provisioning delays. Ideally this should provide sufficient VM resources for the timely launching of new containers in response to sudden workload spikes and avoid costly resource over-provisioning. A further challenge is how to control elastic scaling for both containers and VMs while considering application-level metrics and potential variations in container processing capacity, due to performance interference in shared Cloud data centers. Key to this is the need to optimise performance, availability and costs in a flexible and intelligent manner. This thesis aims to enhance the state-of-the-art in the deployment and elasticity of container-based web applications in geographically distributed Cloud environments, by tacking the above-mentioned problems using meta-heuristics and queuing theory. The thesis makes the following key contributions: 1. it provides an approach for latency-aware failover deployment of container-based web applications across multiple Cloud-based data centers to maintain performance with associated SLOs under normal conditions and in the presence of failures; 2. it provides an approach for dynamic elastic deployment of container-based clusters, both in terms of the quantity and placement across data centers whilst offering trade offs between cost and performance in the context of geographic web workload changes; 3. it offers a cost-efficient, rapid auto-elastic scaling approach for bursty multi-cluster container-based web applications deployed across data centers that scales containers in overloaded situations in a timely and cost-efficient fashion; 4. it presents a two-level elasticity controller algorithm that seamlessly auto-scales at both the container and VM levels based on application-level metrics and queuing-based performance models through estimating the container capacity needed without violating SLOs; 5. it supports dynamic, inter-data center latency aware container scheduling policies for cross-data center clusters that are able to optimise the overall performance, and 6. it presents extensive experiments using case studies based on the container technologies Docker, Docker-Swarm and Kubernetes on the Australia-wide distributed Cloud computing environment (NeCTAR) and international (commercial) cloud data centres