Feedback directed prefetching: Improving the performance and bandwidth-efficiency of hardware prefetchers

High performance processors employ hardware data prefetching to reduce the negative performance impact of large main memory latencies. While prefetching improves perfor-mance substantially on many programs, it can significantly re-duce performance on others. Also, prefetching can significantly increase memory bandwidth requirements. This paper proposes a mechanism that incorporates dynamic feedback into the de-sign of the prefetcher to increase the performance improvement provided by prefetching as well as to reduce the negative per-formance and bandwidth impact of prefetching. Our mecha-nism estimates prefetcher accuracy, prefetcher timeliness, and prefetcher-caused cache pollution to adjust the aggressiveness of the data prefetcher dynamically. We introduce a new method to track cache pollution caused by the prefetcher at run-time. We also introduce a mechanism that dynamically decides where in the LRU stack to insert the prefetched blocks in the cache based on the cache pollution caused by the prefetcher. Using the proposed dynamic mechanism improves average performance by 6.5 % on 17 memory-intensive benchmarks in the SPEC CPU2000 suite compared to the best-performing conventional stream-based data prefetcher configuration, while it consumes 18.7 % less memory bandwidth. Compared to a conventional stream-based data prefetcher configuration that consumes similar amount of memory bandwidth, feedback di-rected prefetching provides 13.6 % higher performance. Our results show that feedback-directed prefetching eliminates the large negative performance impact incurred on some bench-marks due to prefetching, and it is applicable to stream-based prefetchers, global-history-buffer based delta correla-tion prefetchers, and PC-based stride prefetchers. 1