Modelling resource heterogeneities in cloud simulations and quantifying their accuracy

Simulations are often used to evaluate the performance of various scheduling and migration techniques in the context of large computing systems such as clouds and datacenters. To ensure that simulations match the real platform as close as possible, plausible assumptions and accurate statistical models are used in designing simulations; and that could also offer accurate results. However, it is not always possible that similar numerical results would also be achievable in a real cloud test-bed. The reason is that a simulator only abstracts a model and, hence, a system; but does not always reflect the real world scenarios. Therefore, the solution of any research problem using numerical simulation (experimentation) is not just to find a result, but also to ensure the quality and accuracy of the estimated results. CloudSim is largely used in the cloud research community to evaluate the performance of various resource allocation and migration policies. However, resources such as CPU, memory and application heterogeneities are not modelled yet. Moreover, its accuracy is rarely addressed. In this paper, we: (i) describe an extension to CloudSim that offers support for resource (CPU) and application heterogeneities; and (ii) demonstrate several techniques that could be used to measure the accuracy of results obtained in simulations, particularly, in the extended CloudSim. Based on our evaluation, we suggest that the accuracy and precision of the extended version of the CloudSim simulator may be as high as ∼ 98.63% for certain energy and performance efficient resource allocation and consolidation with migration policies in heterogeneous datacenters