Distributed General Matrix Multiply and Add for a 2D Mesh Processor Network

. A distributed algorithm with the same functionality as the single-processor level 3 BLAS operation GEMM, i.e., general matrix multiply and add, is presented. With the same functionality we mean the ability to perform GEMM operations on arbitrary subarrays of the matrices involved. The logical network is a 2D square mesh with torus connectivity. The matrices involved are distributed with non-scattered blocked data distribution. The algorithm consists of two main parts, alignment and data movement of subarrays involved in the operation and a distributed blocked matrix multiplication algorithm on (sub)matrices using only a square submesh. Our general approach makes it possible to perform GEMM operations on non-overlapping submeshes simultaneously. 1 Introduction The level 3 Basic Linear Algebra Subprograms (BLAS) [6] were developed to exploit the memory hierarchy of modern advanced architectures, including vector and RISC-based processors and parallel computers. With optimized level 3 ..