Discovering Scientific Workflows: The myExperiment Benchmarks

Automation in science is increasingly marked by the use of workflow technology. The sharing of workflows through publication mechanisms or repositories supports the verifiability, reproducibility and extensibility of computational experiments. However, the subsequent discovery of workflows remains a challenge, both from a technological and sociological viewpoint. We investigate current practices in workflow sharing, re-use and discovery amongst life scientists chiefly using the Taverna workflow management system. The study draws on two key sources: (i) a survey of researchers drawn from 19 research labs and (ii) an analysis of scientists’ behaviour on the myExperiment social network site, designed to encourage workflow exchange. The results reveal a multi-modal approach to workflow discovery, based on a mix of search on the content of the workflow and its situated context. We go on to develop a benchmark specifically for the evaluation of workflow discovery and to demonstrate it on two example approaches