ElasticMark: An elasticity benchmarking framework for cloud platforms

Elasticity is the unique cost-effective proposition of the cloud. It promises rapid adjustment of resources in response to varying workloads so that the application can meet its QoS objectives with minimal operational expenses. It is a crucial attribute that all commercial cloud providers frequently claim to possess in their offerings. However, existing literature has not yet provided any meaningful and systematic guidance to evaluate the elasticity of the cloud platform from the consumer’s viewpoint. The lack of an elasticity benchmarking framework makes it difficult for the consumer to diagnose and avoid elasticity issues, validate various claims of elasticity and compare the desirability of competing elastic cloud platforms.As such, this thesis proposes a novel consumer-centric elasticity benchmarking framework ElasticMark that reflects the elasticity of the cloud platform as a single figure of merit. It takes the consumer’s perspective on running the benchmark by incorporating her application and workload profiles and then encapsulating the consumer’s business objectives into the elasticity metric based on observations accessible via the cloud APIs. The core framework is comprised of a penalty based elasticity measurement model, a standard workload suite with time-varying workload patterns and a set of guidelines for instantiating an executable benchmark. The measurement model derives the elasticity metric based on financial penalty rates resulting from over-provisioning (unutilized resources) and under-provisioning (inadequate resources) when the cloud-hosted application is exposed to a suite of fluctuating workloads. ElasticMark also includes a novel workload model in order to assist the consumer to generate representative prototypes of her application-specific fine-scale bursty workloads, thus facilitates custom elasticity benchmarking. Furthermore, this framework recommends a set of rigorous techniques to ensure repeatable and valid benchmarking results in presence of the performance unpredictability of the cloud environment. The framework has been validated against a widely-used commercial cloud service to make sure that the elasticity metric is a good reflection of the low-level adaptability characteristics and observed phenomena. It has also been proven effective in comparing and contrasting the elasticity of multiple cloud platforms as well as pinpointing anomalies in their adaptive behaviors