Architecture-Aware Cost Modelling for Parallel Performance Portability

Abstract: Languages for efficient parallel programming need to achieve high per-formance portability in order to harness the power offered by rapidly evolving par-allel architectures. We use a combination of high-level architecture-aware cost mod-elling with a low-level, explicit control of coordination as a programming model to improve performance portability. We explore and quantify the impact of heterogeneity in modern parallel architectures on the performance of parallel programs on a range of clusters of multi-cores, varying in architectural parameters such as processor speed, memory size and interconnection speed. Additionally, we develop several formal cost models and automatically use these architectural characteristics to determine suitable granularity and work placement. The effectiveness of such cost-model-driven manage-ment of parallelism on common-place cluster hardware is demonstrated by measuring the performance of a parallel sparse matrix multiplication, implemented in C+MPI, on a range of heterogeneous architectures. On a cluster with 16 cores, the speedup increases from 6.2, without any cost model, to 9.1, indicating that even a simple, static cost model is effective in adapting the execution to the target architecture and in signi-ficantly improving parallel performance and scalability with negligible overhead.