Memory Forwarding: Enabling Aggressive Layout Optimizations by Guranteeing the Safety of Data Relocation

By optimizing data layout at run-time, we can potentially en-hance the performance of caches by actively creating spatial lo-cality, facilitating prefetching, and avoiding cache conflicts and false sharing. Unfortunately, it is extremely difficult to guarantee that such optimizations are safe in practice on today’s machines, since accurately updating all pointers to an object requires perfect alias information, which is well beyond the scope of the compiler for languages such as C. To overcomethis limitation, we propose a technique called memory forwarding which effectively adds a new layer of indirection within the memory system whenever necessary to guarantee that data relocation is always safe. Because actual forwarding rarely occurs (it exists as a safety net), the mechanism can be implemented as an exception in modern superscalar pro-cessors. Our experimental results demonstrate that the aggres-sive layout optimizations enabled by memory forwarding can re-sult in significant speedups—more than twofold in some cases—by reducing the number of cache misses, improving the effectiveness of prefetching, and conserving memory bandwidth. 1