Data Migration in Distributed Repositories for Collaborative Research

Abstract. In recent years, there has been a growing demand on the required resources in terms of computation and storage to satisfy the needs of scientific experiments and large scale scientific simulations. Scientific applications especially in several areas such as physics, biology, and astronomy have become more complex; and, geographically separated distributed resources have been used to satisfy their immense computational requirements. On the other hand, these applications have become increasingly data intensive such that storage requirements went from giga- to petascale. The distributed nature of the resources made I/O the major bottleneck for end-to-end application performance. Since we need to deal with huge collection of information and distributed data sets, and also these complex applications use heterogeneous resources connected over highly distributed networks, data management middleware has become one of the most important challenges in the area. Therefore, we present a data placement scheduler, for mitigating the data bottleneck in collaborative peta-scale computing systems