Chapter 2 Parallel Computer Architectures

Parallel computers provide great amounts of computing power, but they do so at the cost of increased diÆculty in programming and using them. Certainly, a uniprocessor that was fast enough would be simpler to use. To explain why parallel computers are inevitable and to identify the challenges facing developers of parallel algorithms, programming models, and systems, in this chapter we describe brie y (but in more detail than in Chapter 2) the architecture of both uniprocessor and parallel computers. We will see that while computing power can be increased by adding processing units, memory latency (the irreducible time to access data) is the source of many challenges in both uniprocessor and parallel processor design. Parallel architectures and programming models are not independent. While most architectures can support all programming models, they may not be able to do so eÆciently. An important part of any parallel architecture is any fea-ture that simplies the process of building, testing, and tuning an application. Some parallel architectures put a great deal of eort into supporting a paralle