Quantifying the performance isolation properties of virtualization systems

In recent years, there have been a number of papers comparing the performance of different virtualization environ-ments for x86. These comparisons have typically quantified the overhead of virtualization for one VM compared to a base OS. It has also been common to present data on the scalability of the system in terms of how many identically configured virtual machines can be run on a single physical machine or the performance degradation experienced when multiple VMs are running the same workload.. However, one important aspect of comparing virtualization systems is often not quantified – the degree to which they limit the impact of a misbehaving virtual machine on other virtual machines. Quantifying performance isolation is challenging because the impact can vary based on the type of “misbehavior”. Nevertheless, performance isolation is an essential feature especially in commercial service hosting environments, a key target application of modern virtualization systems, where mutually untrusting VMs run on the same physical machine. In this paper, we present the design of a performance isolation benchmark that quantifies the impact on well-behaved VMs of various categories of extreme resource consumption in a misbehav-ing VM. Our test suite includes six different stress tests- a CPU intensive test, a memory intensive test, a disk inten-sive test, two network intensive tests (send and receive) and a fork bomb. We describe the design of our benchmark suite and present results of testing three flavors of virtualization – VMware (an example of full virtualization), Xe