A study of low power and high performance cache hierarchy for multi-core processor.

The increasing levels of transistor density have enabled integration of an increasing number of cores and cache resources on a single chip. However, power, as a first order design constraint may bring this trend to a dead end. Recently, the primary design objective has been shifted from pursuing faster speed to higher power-performance efficiency. This is also reflected by the fact that design preference has transitioned from fast super-scalar architecture to slower multi-core architecture. Tiled chip multiprocessors (CMPs) have shown unmatched advantages in recent years, and they are very likely to be the mainstream in the future. Meanwhile, increasing number of cores will exert higher pressure on the cache system. Expanding cache storage can ease the pressure but will incur higher static power consumption. More importantly, very large caches in future multi-core systems may not be fully utilised. Under-utilised caches consume static power for no productivity. Off-line profiling of applications to determine optimal cache size and configuration is not practical. This thesis describes dynamic cache reallocation techniques for tiled multi core architectures. We proposed the idea of Break Even number of Misses (BEM). BEM defines, for a given cache configuration and time interval, the maximum number of misses that can be tolerated without increasing the energy delay product. We use BEM as the upper bound to determine a set of thresholds that are used to periodically evaluate the utility of cache. Based on this scheme, we then propose a conservative increase-only resizing method to tune the cache size at tile-level granularity. The increasing only method can be further extended to a dynamic downsizing scheme. In simulations, our tuning scheme can reduce the static power significantly with a very minor degradation of IPC (Instruction Per Cycle). One thing that can be improved from our resizing scheme is the replacement policy. The estimation of cache utility is based on stack-distance which relies on the recency position of a real LRU (least recently used) replacement policy. As is commonly known, LRU is not easy to implement, especially in high associativity caches, which are becoming more commonly used. LRU also suffers from “cache thrashing”, “scan” and “inter-thread interference”. To solve these three problems, we further propose a novel replacement policy, the Effectiveness-Based Replacement policy (EBR) and a refinement, Dynamic EBR (D-EBR), which combines measures of recency and frequency to form a rank sequence inside each set and evict blocks with lowest rank. To evaluate our design, we simulated all 30 applications from SPEC CPU2006 for uni-core system and a set of combinations for 4-core systems, for different cache sizes. The results show that both of them can achieve higher performance with only half the hardware overhead of real LRU. With the help of EBR, we further extend our last level cache resize scheme. We discuss how to estimate equivalent utility of cache using EBR replacement policy rather than LRU, and introduce an EBR based resizing scheme. Since EBR replacement policy is hardware economical and cache thrashing protected, it is more suitable for the utility estimation. Finally, to shorten average cache access latency, we propose the idea of using a private replica region to store useful data replicas. Keeping them close to the requester can significantly reduce average access latency, however, it also reduces effective storage size causing higher cache miss rates. We leverage our EBR based cache utility estimation method to dynamically change the partition based on cache access patterns to achieve a near optimal result.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201