Optimizing bandwidth limited problems using one-sided communication and overlap

This paper demonstrates the one-sided communication used in languages like UPC can provide a significant per-formance advantage for bandwidth-limited applications. This is shown through communication microbenchmarks and a case-study of UPC and MPI implementations of the NAS FT benchmark. Our optimizations rely on aggressively overlapping communication with computation, alleviating bottlenecks that typically occur when communication is iso-lated in a single phase. The new algorithms send more and smaller messages, yet the one-sided versions achieve> 1.9 × speedup over the base Fortran/MPI. Our one-sided versions show an average 15 % improvement over the two-sided versions, due to the lower software overhead of one-sided communication, whose semantics are fundamentally lighter-weight than message passing. Our UPC results use Berkeley UPC with GASNet and demonstrate the scalability of that system, with performance approaching 0.5 TFlop/s on the FT benchmark with 512 processors. 1