Window-based Cost-effective Auto-scaling Solution with Optimized Scale-in Strategy

Auto-scaling is a major way of minimizing the gap between the demand and the availability of the computing resources for the applications with dynamic workloads. Even though a lot of effort has been taken to address the requirement of auto-scaling for the distributed systems, most of the available solutions are application-specific and consider only on fulfilling the application level requirements. Today, with the pay-as-you-go model of cloud computing, many different price plans have been offered by the cloud providers which leads the resource price to become an important decision-making criterion at the time of auto-scaling. One major step is using the spot instances which are more advantageous in the aspect of cost for elasticity. However, using the spot instances for auto-scaling should be handled carefully to avoid its drawbacks since the spot instances can be terminated at any time by the infrastructure providers. Despite the fact that some cloud providers such as Amazon Web Services and Google Compute Engine have their own auto-scaling solutions, they do not follow the goal of cost-effectiveness. In this work, we introduce our auto-scaling solution that is targeted for middle-layers in-between the cloud and the application, such as Karamel. Our work combines the aspect of minimizing the cost of the deployment with maintaining the demand for the resources. Our solution is a rule-based system that is built on top of resource utilization metrics as a more general metric for workloads. Further, the machine terminations and the billing period of the instances are taken into account as the cloud source events. Different strategies such as window based profiling, dynamic event profiling, and optimized scale-in strategy have been used to achieve our main goal of providing a cost-effective auto-scaling solution for cloud-based deployments. With the help of our simulation methodology, we explore our parameter space to find the best values under different workloads. Moreover, our cloud-based experiments show that our solution performs much more economically compare to the available cloud-based auto-scaling solutions.