Latency Tolerance: A Metric for Performance Analysis of Multithreaded Architectures

Multithreaded multiprocessor systems (MMS) have been proposed to tolerate long latencies for communication. This paper provides an analytical framework based on closed queueing networks to quantify and analyze the latency tolerance of multithreaded systems. We introduce a new metric, called the tolerance index, which quantifies the closeness of performance of the system to that of an ideal system. We characterize the latency tolerance with the changes in the architectural and program workload parameters. We show how an analysis of the latency tolerance provides an insight to the performance optimizations of fine grain parallel program workloads. 1 Introduction A multithreaded multiprocessor system (MMS) like TERA [4] and Alewife [3], tolerates long latencies for communication by rapidly switching context to another computation thread, when a long latency is encountered. Multiple outstanding requests for multiple threads at a processor increase the latencies. An informal notion of lat..