Monitoring, Analysing and Predicting Network Performance in Grids

The grid computing paradigm has facilitated the instrumentation of complex, highly-demanding collaborative applications. The technologies that make grid com-puting possible have mostly evolved from parallel and cluster systems. Although this has certainly empowered the grid computing field, part of the heritage has been the perception that required network resources are taken for granted. This is precarious, considering that most grids rely on public IP networks, like the Internet, as the underlying network. This assumption has obstructed the path of grid computing. This thesis aims to improve the performance of grid applications by facilitating network-aware grid scheduling. This is achieved by providing network performance information to grid schedulers, allowing them to adapt to changes in the network. The contribution of this thesis is twofold: a novel approach to network measurement that is particularly suitable for grid environments; and a distributed system that collects and manages these measurements, predicts future network performance, and disseminates this information to schedulers. The accuracy and effectiveness of this system is evaluated on a production grid infrastructure used for e-science applications. The outcomes of this evaluation provide a strong argument for the introduction of network-aware grid schedulers, information systems, and job and resource description standards. i Declaration I declare that the work in this thesis is my own work and has not been submitted either in whole or in part for the award of a higher degree elsewhere. Any sections of the thesis which have been published are clearly identified. Thesis originally submitted on 29 October 2010 Recommended corrections submitted on 19 September 2011 Yehia El-khatib c © Copyright by Yehia El-khatib