Experiences with Data Distribution on NUMA Shared Memory Multiprocessors

The choice of a good data distribution scheme is critical to performance of data-parallel applications on both distributed memory multiprocessors and NUMA shared memory multiprocessors. The high cost of interprocessor communication in distributed memory multiprocessors makes the minimization of communications the predominant issue in selecting data distribution schemes. However, on NUMA multiprocessors other issues such as contention, false sharing and cache affinity also affect performance. In this paper, we present empirical measurements which suggest that these issues cannot be ignored in selecting data distribution schemes. We conclude that existing methodologies used by application programmers and compilers to select data distribution schemes on distributed memory multiprocessors are not suitable for NUMA multiprocessors