Automatic Data Distribution for Massive Parallel Computers

Data distribution is one of the key aspects that a parallelizing compiler for a distributed memory architecture should consider, in order to get efficiency from the system. The cost of accessing local and remote data can be one or several orders of magnitude different, and this can dramatically affect performance. In this report, we propose a novel approach to automatically perform static data distribution. All the constraints related to parallelism and data movement are contained in a single data structure, the Communication-Parallelism Graph (CPG). The problem is solved using a linear 0-1 integer programming model and solver. This allows us to find the best solution for the problem for one-dimensional array distributions. An iterative approach is proposed to solve the multi-dimensional problem. We also show the feasibility of using this approach to solve the problem in terms of compilation time and quality of the solutions generated. 1 Introduction Data distribution is one of the ke..