An automatic tool for benchmark testing of cloud applications

The performance testing of cloud applications is a challenging research topic, due to the multiplicity of different possibilities to allocate application services to Cloud Service Providers (CSPs). Currently available benchmarks mainly focus on evaluating specific services or infrastructural resources offered by different CSPs, but are not always useful to evaluate complete cloud applications and to discover performance bugs. This paper proposes a methodology to define an evaluation performance process, particularly suited for cloud applications, and an automatic procedure to set up and to execute benchmark tests. The methodology is based on the evaluations of two performance indexes, and is validated by presenting a complete case study application, developed within the FP7-EU-SPECS project. The analysis of the measurement results, produced automatically, can help the developer to discover possible bottlenecks and to take actions to improve both the usability and the performance of a cloud application