Program transformation and code generation for developing, modeling, and optimizing GPU programs

The ability to efficiently optimize or re-optimize an algorithm for high performance on a particular processor architecture is crucial in a wide spectrum of engineering and scientific applications. To help bridge the gap between fully automated optimization, which does not reliably produce optimal code, and fully manual optimization, which is inefficient and error-prone, we present three mechanisms facilitating a human-guided, partially automated process for production, optimization, and analysis of high-performance code: (1) formal mechanisms for describing and reasoning about program dependencies and loop nest structure within a programming system providing transformation-based code generation for GPUs and CPUs; (2) a system for constructing customizable, black-box performance models for GPUs; and (3) a visual user interface for program optimization and analysis. We demonstrate how our human-accessible dependency and loop-nest semantics enable reasoning about program correctness in a transformation and optimization procedure for two example scientific computing computations. We show how models created using our hardware-agnostic performance modeling approach can predict execution times accurately enough to select the best-performing of multiple implementation variants of a given computation, demonstrating results for three computations on five different GPUs. We show how our program transformation interface enables access to and navigation of a comprehensive search space of implementation variants for a given computation, as well as comparisons of multiple optimization paths. In doing so, we demonstrate how these architecture-agnostic tools, some of which are already being utilized by a persistent user base, further the goal of facilitating production, optimization, and analysis of high-performance code