Runtime support for object-based message-driven parallel applications on heterogeneous clusters

In the last several years, there has been a growing interest in utilizing accelerator technologies within the realm of high performance computing (HPC). Some of these technologies include the Cell processor (Cell), graphics processing units (GPUs), many integrated core (MIC), the single chip cloud (SCC), and field programmable gate arrays (FPGAs). However, making use of these technologies is typically considered hard for various reasons, including their asynchronous nature, their highly parallel nature and the architecture-specific details involved in programming them. As a result, considerable effort has been put forth in harnessing the computing power of these technologies for more general purpose programming. In this work, we explore (1) how accelerators can be abstracted within a programming model and (2) how an underlying runtime system can assist programmers in making effective use of accelerator technologies. In our approach, the Charm++ programming model and Charm++ runtime system are extended to support the Cell processor and general purpose GPUs (GPGPUs), creating a unified programming model for both accelerators and host cores. Using this single programming model, along with support from an intelligent, adaptive, and proactive runtime system, our work allows applications to make use of various core types (i.e. host, Cell, and GPGPU cores) by dynamically mapping work across the available cores. As an application executes, the runtime system makes use of various load balancing strategies and data management techniques developed as part of this work, allowing the programmer to focus on their application’s goals rather than the implementation of those goals on specific hardware architectures with specific execution models (e.g. SIMD on GPGPUs or streaming on Cell). The resulting code is then portable across a variety of systems, including those that make use of Cell processors and GPGPUs. This dissertation also describes new techniques for extending the runtime system to better integrate accelerator support with various run-time system features. For example, support for a checkpoint/restart fault tolerance mechanism has been integrated into our accelerator extensions, allowing applications making use of accelerators to make use of this fault tolerance scheme with no additional effort by the programmer