Programming Distributed-Memory Architectures The Concept of Message Passing

User explicitly distributes data User explicitly defines communication Compiler has to do no additional work Typically domain or work decomposition is used Typically communication across borders of domains is necessary User defined communication (4)Parallelrechner – Blockkurs im SS2009 Programming Distributed-Memory Architectures The Message Passing Paradigm The same program on each processor/machine (SPMD) Restriction of the general MP model? →No, because processes can be distinguished by their rank (see later) The program is written in a sequential language (FORTRAN/C[++]) All variables are local! → No concept of shared memory Data exchange between processes: Send / Receive messages via appropriate library This is usually the most tedious but also the most flexible way of parallelization Widely accepted message passing standards