Scalable Data Gathering for Real-time Monitoring Systems on Distributed Computing

Real-time monitoring is increasingly becoming impor-tant in various scenes of large scale, multi-site dis-tributed/parallel computing, e.g, understanding behavior of systems, scheduling resources, and debugging applications. Dedicated networks on inter-site communications are rarely available for the monitoring purposes. Therefore, for real-time monitoring systems, reducing communication cost is important to handle a large number of nodes with limited network resources. We implemented a real-time Grid mon-itoring system called VGXP, with techniques for low cost data gathering. It tries to send only diffs to recent data, and adapts to the requested data freshness and tolerable errors to minimize required communication. We evaluate moni-toring overheads of the proposed method on a distributed environment consisting of 8-sites with 500 nodes. In a real-istic setting where the sampling interval is set to 0.5 seconds and the tolerable error to 2%, the CPU usage of the server to gather data from all nodes was 0.2 % and the transfer rate was less than 5kbps. The transfer rate did not exceed 50kbps even if we gather a detailed per-process statistics. 1