Programming Models, Compilers, and Runtime Systems for Accelerator Computing

Accelerators, such as GPUs and Intel Xeon Phis, have become the workhorses of high-performance computing. Typically, the accelerators act as co-processors, with discrete memory spaces. They possess massive parallelism, along with many other unique architectural features. In order to obtain high performance, these features must be carefully exploited, which requires high programmer expertise. This thesis presents new programming models, and the necessary compiler and runtime systems to ease the accelerator programming process, while obtaining high performance. This thesis first presents a mechanism to automatically cater to out-of-card problem sizes, and achieve multi-GPU scalability. Next, the thesis presents a heterogeneous MapReduce programming system to cater to out-of-node datasizes. The system proposes novel program constructs, and a MapReduce domain-specific optimizing compiler. It also presents a scheduling mechanism to reduce load imbalance across CPUs and GPUs. Lastly, the thesis presents formalization of structured control flow graphs. We present a mechanism to convert any unstructured CFG into a structured CFG without facing code explosion. This mechanism can be used to reduce the impact of control flow divergence on accelerators that contain SIMD execution units