Development of a GT4-based Resource Broker Service: an application to on-demand weather and marine forecasting

The discovery and selection of needed resources, taking into account optimization criteria, local policies, computing and storage availability, resource reservations, and grid dynamics, is a technological challenge in the emerging technology of grid computing. The Condor Project’s ClassAd language is commonly adopted as a “lingua franca” for describing grid resources, but Condor itself does not make extensive use of Web Services. In contrast, the strongly service-oriented Globus Toolkit is implemented using the web services resource framework, and offers basic services for job submission, data replica and location, reliable file transfers and resource indexing, but does not provide a resource broker and matchmaking service. In this paper we describe the development of a Resource Broker Service based on the Web Services technology offered by the Globus Toolkit version 4 (GT4). We implement a fully configurable and customizable matchmaking algorithm within a framework that allows users to direct complex queries to the GT4 index service and thus discover any published resource. The matchmaking algorithm supports both the native simple query form and the Condor ClassAd notation. We achieve this flexibility via a matchmaking API java class framework implemented on the extensible GT4 index service, which maps queries over ClassAds in a customizable fashion. We show an example of the proposed grid application, namely an on demand weather and marine forecasting system. This system implements a Job Flow Scheduler and a Job Flow Description Language in order to access and exploit shared and distributed observations, model software, and 2D/3D graphical rendering resources. The system combines GT4 components and our Job Flow Scheduler and Resource Broker services to provide a fully grid-aware system