Middleware for Astronomical Data Analysis Pipelines

In this paper the authors describe the approach to research, develop, and evaluate prototype middleware tools and architectures. The developed tools can be used by scientists to compose astronomical data analysis pipelines easily. They use the SuperMacho data pipelines as example applications to test the framework. they describe their experience from scheduling and running these analysis pipelines on massive parallel processing machines. they use MCR a Linux cluster machine with 1152 nodes and Luster parallel file system as the hardware test-bed to test and enhance the scalability of the tools