The SpiceC Parallel Programming System

As parallel systems become ubiquitous, exploiting parallelism becomes crucial for improving application performance. However, the complexities of developing parallel software are major challenges. Shared memory parallel programming models, such as OpenMP and Thread Building Blocks (TBBs), offer a single view of the memory thereby making parallel programming easier. However, theysupport limited forms of parallelism. Distributed memory programming models,such as the Message Passing Interface (MPI), support more parallelism types; however, their low level interfaces require great deal of programming effort.This dissertation presents the SpiceC system that simplifies the task of parallel programming while supporting different forms of parallelism and parallel computing platforms. SpiceC provides easy to use directives to express different forms of parallelism, including DOALL, DOACROSS, and pipelining parallelism. SpiceC is based upon an intuitive computation model in which each thread performs its computation in isolation from other threads using its private space and communicates with other threads via the shared space. Since, all data transfers between shared and private spaces are explicit, SpiceC naturally supports both shared and distributed memory platforms with ease. SpiceC is designed to handle the complexities of real world applications.The effectiveness of SpiceC is demonstrated both in terms of delivered performance and the ease of parallelization for applications with the following characteristics.Applications that cannot be statically parallelized due to presence of dependences, often contain large amounts of input dependent and dynamic data level parallelism. SpiceC supports speculative parallelizationfor exploiting dynamic parallelism with minimal programming effort. Applications that operate on large data sets often make extensive use of pointer-based dynamic data structures. SpiceC provides support for partitioning dynamic data structures across threads and then distributing the computation among the threads in a partition sensitive fashion. Finally, due to large input sizes, many applications repeatedly performI/O operations that are interspersed with the computation. While traditionalapproach is to execute loops contain I/O operations serially, SpiceC introduces support for parallelizing computations in the presence of I/O operations. Finally, this dissertation demonstrates that SpiceC can handle the challengesposed by the memory architectures of modern parallel computing platforms.The memory architecture impacts the manner in which data transfers betweenprivate and shared spaces are implemented. SpiceC does not place thethe burden of data transfers on the programmer. Therefore portability of SpiceCto different platforms is achieved by simply modifying the handling of data transfers by the SpiceC compiler and runtime. First, it is shown how SpiceC can be targeted to shared memory architectures both with and without hardware support for cache coherence. Next it is shown how accelerators such as GPUs present in heterogeneous systems are exploited by SpiceC. Finally, the ability of SpiceC to exploit the scalability of a distributed-memory system, consisting of a cluster of multicore machines, is demonstrated