Dynamic workflow adaptation over adaptive infrastructures

There is emerging interest in many scientific disciplines to deal with “dynamic” data, arising from sensors and scientific instruments, which require workflow graphs that can be dynamically adapted – as new data becomes available. Additionally, the elastic nature of many Cloud environments subsequently enable such dynamic workflow graphs to be enacted more efficiently. One of the challenges of scientific workflows is that they must be designed with the needed level of dynamism to take account of the availability of data and the variability of the execution environment, which can be dynamically scaled out based on demand (and budget). In this paper, we present a novel approach for specifying scientific workflows with the two main requirements of: (i) dynamic / adaptive workflow structure well suited for and responsive to change, and (ii) support for large-scale and variable parallelism. We utilise the superscalar pipeline as a model of computation and the well-known Montage workflow for illustrating our approach