On Architecture Independent Design and Analysis of Parallel Programs

The presence of a universal machine model for serial algorithm design, namely the von Neumann model, has been one of the key ingredients of the success of uniprocessors. The presence of such a model makes it possible for an algorithm to be ported across a wide range of uniprocessors efficiently. Universal models proposed for parallel computers however tend to be highly limited in the range of parallel computers they can efficiently cover. Consequently, portability of parallel programs is attained at the expense of loss of efficiency. In this paper, we explore desirable and attainable properties of universal models of architecture independent parallel program design. We study various models that have been proposed, classify them based on important machine parameters and study their limitations. We examine the role of aggregate communication operations. Based on this, we present a framework for analyzing communication overheads in parallel programs. We show that a wide variety of program..