A Framework for Benchmark Performance Analysis

Introduction The conclusions drawn from a benchmark study of computer performance depend not only on the basic timing results obtained, but also on the way there are interpreted and converted into performance figures. The choice of the performance metric, may itself influence the conclusions. For example, do we want the computer that generates the most Mflop/s (or has the highest speedup), or the computer that solves the problem in the least time? It is now well known that high values of the first metrics do not necessarily imply the second property. This confusion can be avoided by choosing a more suitable metric that reflects solution time directly, for example either the temporal or simulation performance, defined below. This issue of the sensible choice of performance metric is becoming increasing important with the advent of massively parallel computers which have the potential of very high Mflop rates, but have much more limited potential for reducing solution time. Th