Design and Implementation of a Scalable Network Monitoring System

Monitoring systems give network administrators a better view and understanding of their networks. Amongst their many uses, they can be used to audit computing assets, profile resource usage, and pinpoint security problems. Current monitoring systems have not really explored the limits of monitoring scalability, preferring to focus on other important issues such as reliability and node discovery. We present a monitoring system that scale to over 100000 nodes. It has minimal local and global overhead, and maintains integrity in the face of transient network failure. Through a hierarchal organisation, our monitoring system can operate in multiple administrative zones. Since we did not have a large fleet of machines at our disposal, we simulated a large fleet of machines, upon which a smaller network of real machines was overlayed. This system was the testbed for our scal