Flexible Timing Simulation of Multiple-Cache Configurations

Abstract As the gap between processor and memory speeds increases, cache performance becomes more critical to overall system performance. Behavioral cache simulation is typically used early in the design cycle of new processor/cache configurations to determine the performance of proposed cache configurations on target workloads. However, behavioral cache simulation does not account for the latency seen by each memory access. The Latency-Effects (LE) cache model presented in this paper accounts this nominal latency as well as the additional latencies due to trailing-edge effects, bus width considerations, port conflicts, and the number of outstanding accesses that a cache allows before it blocks. We also extend the LE cache model to handle the latency effects of moving data among multiple caches. mlcache, a new, easily configurable and extensible tool, has been built based on the extended LE model. We show the use of mlcache in estimating the performance of traditional and novel cache configurations, including odd/even, 2-level, Assist, Victim, and NTS caches. We also show how the LE cache timing model provides more useful, realistic performance estimates than other possible behavioral-level cache timing models. Keywords: cache timing simulation model evaluation Introduction Cache performance becomes ever more critical to overall system performance as the gap between processor and memory speed increases. The performance of a particular cache configuration depends not only on the miss ratio incurred during the execution of a particular workload but also on where in the program's execution the misses occur and the latency of each miss. However, useful timing simulation of caches is typically unavailable until late in the design stage. Using today's behavioral simulators, simple, traditional caches are evaluated early in the design cycle; however, novel cache designs are often not considered since they are difficult to model. The issue of providing more useful cache timing simulation analysis early in the design cycle has been addressed by the Latency-Effects (LE) cache model [Tam96], which incorporates latency-adding effects into a behavioral-level simulation, particularly trailing-edge effects, bus width considerations, the effects of port conflicts, and the number of outstanding accesses that a cache can handle before blocking. Existing methods of modifying behavioral cache simulators to incorporate timing effects include adjusting the total cycle count reported by a perfect cache simulation by adding an estimated number of cycles due to cache misses (the adjusted model) or assigning a nominal leading-edge penalty to each miss as it occurs (a model we will refer to as LEnominal). To illustrate the advantages of the LE cache model, we will compare the LE cache model's results to the results of using these other models. 1. This research was funded in part by a gift from IBM