Runtime Models for Extreme Scale Systems

Future extreme-scale systems are expected to contain homogeneous and heterogeneous many-core processors, with up to $O(10^3)$ cores per node and $O(10^6)$ nodes overall. Effective combination of inter-node and intra-node parallelism is recognized to be a major software challenge for such systems. Dynamic task parallelism, which forms the basis of our research, has proven to be an effective execution model enabled by efficient runtime systems for scalable performance on shared-memory programs. This work builds on the Habanero-C task parallel language. We focus on runtime execution models for scalable locality aware task scheduling and synchronization at both intra-node and inter-node levels. A scalable runtime communication system is a critical requirement for extreme-scale computing. We present our runtime communication system, called HC-COMM, which can integrate any popular communication system, such as MPI, with dynamic asynchronous tasks. The runtime uses dedicated computation and communication cores for scalable performance. We introduce the HC-MPI execution model, where all MPI calls are treated as asynchronous tasks, thereby enabling unified handling of messages and tasking constructs. HC-MPI provides easy portability of existing MPI applications. We also introduce the HAPGNS programming model that uses the HC-COMM runtime to extend shared-memory data flow programming onto distributed-memory. In this model, the user does not require any knowledge of MPI. We believe locality of computation and data will be critical for higher performance and lower energy resulting from data reuse on faster memories and redundant communication avoidance. Our locality optimization framework can enable performance experts to optimize existing applications via tuning operations. These optimizations aim to benefit from spatial and temporal task locality using runtime co-scheduling of tasks on a hierarchical place tree construct which models the system memory hierarchy. In the HAPGNS model, user provided data distribution functions act as data locality directives at inter-node level. Finally, we present efficient algorithms to enable the Phaser synchronization model on long running tasks. We have extended the Phaser construct to support unified collective operations at both inter-node and intra-node levels. Our experimental results on current large scale systems serve to validate the design and implementation of our runtime system for extreme-scale computing