PTG: an abstraction for unhindered parallelism

Increased parallelism and use of heterogeneous computing resources is now an established trend in High Performance Computing (HPC), a trend that, looking forward to Ex-ascale, seems bound to intensify. Despite the evolution of hardware over the past decade, the programming paradigm of choice was invariably derived from Coarse Grain Paral-lelism with explicit data movements. We argue that message passing has remained the de facto standard in HPC because, until now, the ever increasing challenges that application de-velopers had to address to create efficient portable applica-tions remained manageable for expert programmers. Data-flow based programming is an alternative approach with significant potential. In this paper, we discuss the Pa-rameterized Task Graph (PTG) abstraction and present the specialized input language that we use to specify PTGs in our data-flow task-based runtime system, PaRSEC. This lan-guage and the corresponding execution model are in contrast with the execution model of explicit message passing as well as the model of alternative task based runtime systems. The Parameterized Task Graph language decouples the expres-sion of the parallelism in the algorithm from the control-flow ordering, load balance, and data distribution. Thus, programs are more adaptable and map more efficiently on challenging hardware, as well as maintain portability across diverse architectures. To support these claims, we discuss the different challenges of HPC programming and how PaR-SEC can address them, and we demonstrate that in today’s large scale supercomputers, PaRSEC can significantly out-perform state-of-the-art MPI applications and libraries, a trend that will increase with future architectural evolution. 1