Analysis of High Performance Scientific Programming Workflows

Substantial time is spent on building, optimizing and maintaining large-scale software that is run on supercomputers. However, little has been done to utilize overall resources efficiently when it comes to including expensive human resources. The community is beginning to acknowledge that optimizing the hardware performance such as speed and memory bottlenecks contributes less to the overall productivity than does the development lifecycle of high-performance scientific applications. Researchers are beginning to look at overall scientific workflows for high performance computing. Scientific programming productivity is measured by time and effort required to develop, configure, and maintain a simulation experiment and its constituent parts, together with the time to get to the solution when the programs are executed. There is no systematic framework by means of which scientific programming productivity of the available tools can be evaluated. We propose an evaluation approach that compares programming workflows to identify productivity bottlenecks and suboptimal paths as well as productivity gains. Based on a set of predefined criteria we can evaluate both short-term and long-term productivity criteria. We use these results to suggest improvements to the programming environment or tools. This thesis includes three studies of scientific programming workflows: 1) We apply our evaluation approach to two case studies involving the use of numerical libraries. 2) We evaluate GPU programming models using software engineering complexity metrics. 3) We evaluate use of a high level directive based programming model and a source-to-source compiler with respect to productivity of programming FPGAs using a computational chemistry code. We compare the programmability and performance of the FPGA port with the GPU port of the same code