Semantic Based Data Collection for Large Scale Cloud Systems

Current tools for monitoring cloud systems are designed for physical servers and are not intended to handle rapid elastic-ity or dynamic behaviour while operating at scale. Though current monitoring tools can be applied to small cloud sys-tems, the volume of data and computational overhead asso-ciated with their operation render them unsuitable for large scale cloud deployments. The metrics obtained by current solutions also lack a machine readable structure, limiting the ability of both software and humans to interpret the data. As cloud adoption continues, the scale and complexity of cloud systems will present significant challenges to current tools. This paper proposes a scalable distributed data col-lection system which forms the basis of a cloud monitor-ing system. Utilising technologies from the semantic web, our architecture generates a machine readable overview of a cloud system without the need for an additional dedicated monitoring system. We present an exemplar implementation of our architecture written using the Python programming language and perform an evaluation demonstrating its abil-ity to provide scalable data collection services fit for cloud computing