Collabra: A Xen Hypervisor Based Collaborative Intrusion Detection System

Abstract—In this paper, we introduce Collabra, a distributed intrusion detection platform based on Xen hypervisors to maintain the security of the cloud based on virtualized network. While the concept of virtual machine monitor (VMM) signifies implementing an abstraction layer between the underlying host and the guest operating system (OS) to enforce security, its kernel is required to be free of vulnerabilities that intruders can use to compromise the host. In Xen, guest applications make resource requests through the hyper-call API to transfer the privilege to the VMM kernel for executing privileged operations. On a cloud scale, there exist hundreds of VM networks and thousands of guest operating systems (OSes) running on virtual domains. There is every possibility of intruders trying to misuse the hyper-call interface to compromise guest OS kernels and finally the host OS kernel itself. Sophisticated attacks can be launched in the distributed and collaborative style thereby bypassing most current intrusion detection systems. Collabra acts as a filtering layer which is completely integrated with every VMM. It scans through each call by incorporating integrity checking and collaborative detection mechanisms. It exists in multiple instances, and acts concurrently over a VMM network interacting with other instances to detect (possibly collaborative) attacks and prevent illicit access to the VMM and the host. An admin version of Collabra exists on a privileged domain in the VM network to perform filtering of malicious add-ons to hyper-calls at the guest OS level itself before routing the call to the VMM. Keywords-Virtual machine; distributed intrusion detection system; intrusion detection architecture; cloud computing; hyper-call I