Appears in ACM Performance Evaluation Review 30(4):19–26 Combining Active and Passive Network Measurements to Build Scalable Monitoring Systems on the Grid

Because the network provides the wires that connect a grid, understanding the performance provided by a network is crucial to achieving satisfactory performance from many grid applications. Monitoring the network to predict its performance for applications is an effective solution, but the costs and scalability challenges of actively injecting measurement traffic, as well as the information access and accuracy challenges of using passively collected measurements, complicate the problem of developing a monitoring solution for a global grid. This paper is a preliminary report on the Wren project, which is focused on developing scalable solutions for network performance monitoring. By combining active and passive monitoring techniques, Wren is able to reduce the need for invasive measurements of the network without sacrificing measurement accuracy on either the WAN or LAN levels. Specifically, we present topology-based steering, which dramatically reduces the number of measurements taken for a system by using passively acquired topology and utilization to select the bottleneck links that require active bandwidth probing. Furthermore, by using passive measurements while an application is running and active measurements when none is running, we preserve our ability to offer accurate, timely predictions of network performance, while eliminating additional invasive measurements.