Towards Workflow Benchmarking: Open Research Challenges

Abstract: Workflow Management Systems (WfMS) provide platforms for delivering complex service-oriented applications that need to satisfy enterprise-grade quality of service requirements such as dependability and scalability. The performance of these applications largely depends on the performance of the WfMS supporting them. Com-paring the performance of different WfMSs and optimizing their configuration requires that appropriate benchmarks are made available. In this position paper we make the case for benchmarking the performance of WfMSs that are compliant with the Busi-ness Process Model and Notation 2.0 standard and explore most of the challenges that one must tackle when constructing such benchmarks.