Integrating software distributed shared memory and message passing programming

Software Distributed Shared Memory (SDSM) systems provide programmers with a shared memory programming environment across distributed memory architectures. In contrast to the message passing programming environment, an SDSM can resolve data dependencies within the application without the programmer having to explicitly specify the required communications. Such ease-of-use is, however, provided at a cost to performance. This thesis considers how the SDSM programming model can be combined with the message passing programming model with the goal of avoiding these additional costs when a message passing solution is straightforward. To pursue the above goal a new SDSM library, named Danui, has been developed. The SDSM manages a shared address space in units of pages. Consistency for these pages is maintained using a variety of home-based protocols. In total, the SDSM includes seven different barrier implementations, five of which support home-migration. Danui is designed with portability and modularity in mind. It is written using the standard Message Passing Interface (MPI) to perform all underlying SDSM related communications. MPI was used both to permit the subsequent exploration of user-level MPI/SDSM programming, and also to enable the SDSM library to exploit optimised MPI implementations on a variety of cluster interconnects. A detailed analysis of the costs associated with the various SDSM operations is given. This analysis characterizes each SDSM operation based upon a number of factors. For example, the size of an SDSM page, the number of modifications made to shared memory, the choice of barrier used etc. This information is then used as a guide to determine which parts of an SDSM program would benefit most if replaced by direct calls to MPI. The integration of the shared memory and message passing (SMMP) programming models within Danui is discussed in detail. It is shown that a na{u00EF}ve integration of the SDSM and MPI programming models leads to memory consistency problems when MPI transfers occur to or from memory that is part of the software managed shared address space. These problems can, however, be overcome by noting the semantics of the shared memory and message passing paradigms, and introducing a few extensions to the MPI interface