Optimising Shared Reduction Variables in MPI Programs

Abstract. CFL (Communication Fusion Library) is an experimental C++ library which supports shared reduction variables in MPI programs. It uses overloading to distinguish private variables from replicated, shared variables, and automatically introduces MPI communication to keep replicated data consistent. This paper concerns a simple but surprisingly effective technique which improves performance substantially: CFL operators are executed lazily in order to expose opportunities for run-time, context-dependent, optimisation such as message aggregation and operator fusion. We evaluate the idea using both toy benchmarks and a `production ' code for simulating plankton population dynamics in the upper ocean. The results demonstrate the software engineering benefits that accrue from the use of the library and show that performance close to that of manually optimised code can be achieved automatically in many cases. 1 Introduction In this paper we describe an experimental abstract data type for representing shared variables in SPMD-style MPI programs. The operators of the abstract data type have a simple and intuitive semantics and hide any required communication. Although there are some interesting issues in the design of the library, the main contribution of this paper is to show how lazy evaluation can expose run-time optimisations that may be difficult, or even impossible, to spot using conventional compile-time analysis. The paper makes the following contributions:- We present a simple and remarkably useful prototype class library whic