How to Choose the Grain Size of a Parallel Computer

Designers of parallel computers have to decide how to apportion a machine's resources between processing, memory, and communication. How these resources are apportioned determine the grain and balance of the resulting machine. Often, these design decisions are made according to rules of thumb which often lead to unoptimized designs. This paper presents an analytical framework upon which designers can reason about the design space of parallel computers, and make design decisions based on analysis. The framework is grounded upon the notion of cost-e ective design. It focuses on the needs of applications and tries to identify machines that will execute these applications e ciently. This type of optimization is made di cult by the fact that the application domain may be diverse, each application demanding very di erent resources for e ciency. This paper makes three contributions in the context of parallel computer design. First, it provides an analytical framework based on a \blc mpP " machine characterization that considers processing, memory, local and global communication, and latency as separate machine resources. This characterization is unique because it accounts for localityby considering local and global communication separately. Second, the model predicts that general-purpose parallel computers are realizable, but may be possible only at extremely high machine budgets. Finally, the model shows that the amount of memory architected into current production parallel machines is suited for machines with 10s to low 100s of processors. These machines are not cost-e ective for moderately or massively parallel systems unless astronomical problem sizes are desirable.