Improving Cache Power EÆciency with an Asymmetric Set-Associative Cache

Data caches are widely used in general-purpose pro-cessors as a means to hide long memory latencies. Set-associativity in these caches helps programs avoid performance problems due to cache mapping con icts. Many programs, however, need high associativity for only some of their frequently-referenced addresses and tolerate much lower associativity for the remainder of the references. With this variability in mind, this pa-per proposes an asymmetric cache structure in which the size of each way can be dierent. The ways of the cache are dierent powers of two, and allow for a \tree-structured " cache in which extra associativity can be shared. We accomplish this by having two cache blocks from the large ways align with individual cache blocks in the smaller ways. This structure achieves miss rates comparable (on average 3 % better for SPEC2000) to a conventional cache of the same size and associativity. Most notably, the asymmetric cache has the nice prop-erty that accesses hit in the smaller ways can immedi-ately terminate accesses to larger ways so that power can be saved. For the SPEC2000 benchmarks, we found cache energy per access was reduced by 17 % on average.