MPC: A Unified Parallel Runtime for Clusters of NUMA Machines

Abstract. Over the last decade, Message Passing Interface (MPI) has become a very successful parallel programming environment for dis-tributed memory architectures such as clusters. However, the architec-ture of cluster node is currently evolving from small symmetric shared memory multiprocessors towards massively multicore, Non-Uniform Memory Access (NUMA) hardware. Although regular MPI implemen-tations are using numerous optimizations to realize zero copy cache-oblivious data transfers within shared-memory nodes, they might prevent applications from achieving most of the hardware’s performance simply because the scheduling of heavyweight processes is not flexible enough to dynamically fit the underlying hardware topology. This explains why several research efforts have investigated hybrid approaches mixing mes-sage passing between nodes and memory sharing inside nodes, such as MPI+OpenMP solutions [1,2]. However, these approaches require lots of programming efforts in order to adapt/rewrite existing MPI applications. In this paper, we present the MultiProcessor Communications envi-ronnement (MPC), which aims at providing programmers with an effi-cient runtime system for their existing MPI, POSIX Thread or hybrid MPI+Thread applications. The key idea is to use user-level threads in-stead of processes over multiprocessor cluster nodes to increase scheduling flexibility, to better control memory allocations and optimize scheduling of the communication flows with other nodes. Most existing MPI applications can run over MPC with no modification. We obtained substantial gains (up to 20%) by using MPC instead of a regular MPI runtime on several scientific applications.