A resource provisioning system for scientific workflow applications

UnrestrictedThe development of grid and workflow technologies has enabled complex, loosely coupled scientific applications to be executed on distributed resources. Many of these applications consist of large numbers of short-duration tasks whose runtimes are heavily influenced by delays in the execution environment. Such applications often perform poorly on the grid because of the large scheduling overheads commonly found in grid environments. In this work we address the performance issues of these applications through the use of resource provisioning techniques. We present a provisioning system based on multi-level scheduling that improves workflow runtime by reducing task scheduling overheads, by reserving resources for the exclusive use of the application, and by giving users control over scheduling policies. This system is shown to reduce the runtime of fine-grained workflow applications by as much as 90% compared to traditional methods