Speculative Vectorization with Selective Replay

While industry continues to develop SIMD vector ISAs by providing new instructions and wider data-paths, modern SIMD architectures still rely on the programmer or compiler to transform code to vector form only when it is safe. Limitations in the power of a compiler’s memory alias analysis and the presence of infrequent memory data dependences mean that whole regions of code cannot be safely vectorised without risking changing the semantics of the application, restricting the available performance. We present a new SIMD architecture to address this issue, which relies on speculation to identify and catch memory- dependence violations that occur during vector execution. Once identified, only those SIMD lanes that have used erroneous data are replayed; other lanes, both older and younger, keep the results of their latest execution. We use the compiler to mark loops with possible cross-iteration dependences and safely vectorise them by executing on our architecture, termed selective-replay vectorisation (SRV). Evaluating on a range of general-purpose and HPC benchmarks gives an average loop speedup of 2.9×, and up to 5.3× in the best case, over already-vectorised code. This leads to a whole-program speedup of up to 1.19× (average 1.06×) over already-vectorised applications