Intelligent dynamic caching for large data sets in a grid environment

Present and future distributed applications need to deal with very large PetaBytes (PB) datasets and increasing numbers of associated users and resources. The emergence of Grid-based systems as a potential solution for large computational and data management problems has initiated significant research activity in the area. Grid research can be divided into two at least areas: Data Grids and Computational Grids. The aims of Data Grids are to provide services for accessing, sharing and modifying large databases, the aims of Computational Grids are to provide services for sharing resources. The considerable increase in data production and data sharing within scientific communities has created the need for improvements in data access and data availability. It can be argued the problems associated with the management of very large datasets are not well serviced by current approaches. The thesis concentrates on one of the areas concerned in the access to distributed very large databases on Grid resources. To this end, it presents the design and implementation of partial replication system and a Grid caching system that mediates access to distributed data. Artificial intelligent (AI) techniques such as a neural network (NN) have been used as a prediction element of the model to determine user requirements by analysing the past history of the user. Hence, this thesis will examine the problems surrounding the manipulation of very large data sets within a Grid-like environment The goal is the development of a prototype system that will enable both effective and efficient access to very large datasets, based on the use of a caching model.EThOS - Electronic Theses Online ServiceGBUnited Kingdo