Improving communication performance of sparse linear algebra for an atomistic simulation application

The design of modern parallel machines leads to powerful machines, but with complex architectures and hierarchical topologies. As a result, communication overheads associated with hardware asymmetry and interconnection network increase. In order to achieve scalable performances on these machines, it is essential to reduce communication costs on parallel applications such as CP2K. From computational chemistry domain, CP2K is a real-world parallel application that performs atomistic and molecular simulations. A linear-scaling density functional theory implementation based on an efficient sparse linear algebra kernel allows CP2K to simulate a million of atoms. Since this kernel is communication bound, the hardware asymmetry and interconnection network of current machines leads to a slowdown on CP2K performance for large number of processes. In this paper, we introduce a heuristic based process mapping to reduce the communication costs. It takes into account the machine topology and the sparse linear algebra kernel communication pattern to map processes over the machine. Results show that our process mapping provides on average a performance improvement of 50% when compared with the default implementation of CP2K