PANORAMA: An Approach to Performance Modeling and Diagnosis of Extreme Scale Workflows

Computational science is well established as the third pillar of scientific discovery and is on par with exper-imentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computa-tions alike, the complexity of managing the compute and data analysis tasks has grown beyond the capa-bilities of domain scientists. Thus, workflow man-agement systems are absolutely necessary to ensure current and future scientific discoveries. A key re-search question for these workflow management sys-tems concerns the performance optimization of com-plex calculation and data analysis tasks. The cen-tral contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific work-flows. This approach integrates extreme-scale sys-tems testbed experimentation, structured analytical modeling, and parallel systems simulation into a com-prehensive workflow framework called Pegasus for un-derstanding and improving the overall performance of complex scientific workflows