Prefetching Complex Access Patterns with Deep Learning

he Von Neumann bottleneck is a persistent problem in computer architecture, causing stalls and wasted CPU cycles. The Von Neumann bottleneck isparticularly relevant for memory-intensive workloads whose working set does not fit into the microprocessor’s cache and hence memory accesses suffer the high access latency of DRAM. One technique to address this bottleneck is to prefetch data from memory into on-chip caches. While prefetching has proven successful, for simple access patterns such as strides, existing prefetchers are incapable of providing benefit for applications with complex, irregular access patterns. A neural network-based prefetcher shows promise for these challenging workloads. We provide an understanding of what type of memory access patterns an LSTM neural network can learn by studying its effectiveness on a suite of microbenchmarks with well-characterized memory access patterns, and perform a parameter sensitivity analysis to identify the most important model parameters. We achieve over 95% accuracy on the microbenchmarks and find a strong relationship between lookback (history window) size and the ability of the model to learn the pattern. We find also an upper limit on the number of concurrent distinct memory access streams that can be learned by a model of a given size