Cache design strategies for efficient adaptive line placement

Efficient memory hierarchy design is critical due to the large difference between the speed of the processors and the memory. In this context, cache memories play a crucial role bridging this gap. Cache management has become even more significant due to the appearance of chip multiprocessors (CMPs), which imply larger memory bandwidth requirements and greater working sets of many emerging applications, and which also need a fair and efficient distribution of cache resources between the cores in a single chip. This dissertation aims to analyze some of the problems commonly found in modern caches and to propose cost-effective solutions to improve their performance. Most of the approaches proposed in this Thesis reduce cache miss rates by taking advantage of the different levels of demand cache sets may experience. This way, lines are placed in underutilized cache blocks of other cache sets if they are likely to be reused in the near future and there is no enough space in their native cache set. When this does not suffice, this dissertation proposes to modify in a coordinated way the insertion policies of oversubscribed sets. Hence, our proposals retain the most useful part of the working set in the cache while discarding temporary data as soon as possible. These ideas, initially developed in the context of last-level caches (LLCs) in single core systems, are successfully adapted in this Thesis to first-level caches and multicore systems. Regarding first-level caches, a novel design that allows to dynamically allocate banks to the instruction or the data cache depending on their degree of pressure is presented. As for multicore systems, our designs are firstly provided with thread-awareness in shared caches in order to give a particular treatment to each stream of requests depending on its owner. Finally, we explore the sharing of resources by means of the spilling of lines among private LLCs in CMPs using several innovative features such as a neutral state, which prevents caches from taking part in the spilling mechanism if this could be harmful, variable granularities for the management of the caches, or the coordinated management of the cache insertion policy. Throughout this process we have used a simple and cost-effective metric to track the state of each cache set called Set Saturation Level (SSL). Finally, it is worthy to point out that our approaches are very competitive and often outperform many of the most recent techniques in the field, despite they imply really small storage and power consumption overheads