HTTP Redirection for Replica Catalogue Lookups in Data Grids

Abstract 1 Data distribution and replication in distributed systems require special purpose middleware tools for accessing replicated data. Data Grids, special forms of a systems distributed over wide-are networks, need to handle data management issues like distribution and replication of large amounts of data in the Tera-and Petabyte scale. Replica catalogues are used for cataloging and locating replicated files in distributed sites all around the globe. We present a novel and scalable approach for distributing a replica catalogue and resolving file location information by using HTTP redirection. HTTP redirection servers managing local file catalogues allow for greater flexibility and local file management autonomy whereas a global replica catalogue provides the necessary mapping of logical files to individual sites. By distributing the catalogues a site can autonomously move files for load balancing within a site without notifying a global replica catalogue. Our approach scales well to a large number of sites and file entries and thus establishes powerful middleware service. We present the design and implementation of our catalogue redirection servers and report on promising experimental results.