RFVP: Rollback-Free Value Prediction with Safe-to-Approximate Loads

This paper aims to tackle two fundamental memory bottle-necks: limited off-chip bandwidth (bandwidth wall) and long access latency (memory wall). To achieve this goal, our ap-proach exploits the inherent error resilience of a wide range of applications. We introduce an approximation technique, called Rollback-Free Value Prediction (RFVP). When certain safe-to-approximate load operations miss in the cache, RFVP predicts the requested values. However, RFVP never checks for or recovers from load value mispredictions, hence avoid-ing the high cost of pipeline flushes and re-executions. RFVP mitigates the memory wall by enabling the execution to con-tinue without stalling for long-latency memory accesses. To mitigate the bandwidth wall, RFVP drops some fraction of load requests which miss in the cache after predicting their values. Dropping requests reduces memory bandwidth con-tention by removing them from the system. The drop rate then becomes a knob to control the tradeoff between perfor-mance/energy efficiency and output quality. For a diverse set of applications from Rodinia, Mars, and NVIDIA SDK, employing RFVP with a 14KB predictor per streaming multiprocessor (SM) in a modern GPU delivers, on average, 40 % speedup and 31 % energy reduction, with aver-age 8.8 % quality loss. With 10 % loss in quality, the benefits reach a maximum of 2.4 speedup and 2.0 energy reduc-tion. As an extension, we also evaluate RFVP’s latency ben-efits for a single core CPU. For a subset of the SPEC CFP 2000/2006 benchmarks that are amenable to safe approxima-tion, RFVP achieves, on average, 8 % speedup and 6 % energy reduction, with 0.9 % average quality loss. 1