Stochastic Optimization of Floating-Point Programs with Tunable Precision

The aggressive optimization of floating-point computations is an important problem in high-performance computing. Unfortunately, floating-point instruction sets have complicated semantics that of-ten force compilers to preserve programs as written. We present a method that treats floating-point optimization as a stochastic search problem. We demonstrate the ability to generate reduced precision implementations of Intel’s handwritten C numeric library which are up to 6 times faster than the original code, and achieve end-to-end speedups of over 30 % on a direct numeric simulation and a ray tracer by optimizing kernels that can tolerate a loss of preci-sion while still remaining correct. Because these optimizations are mostly not amenable to formal verification using the current state of the art, we present a stochastic search technique for characterizing maximum error. The technique comes with an asymptotic guaran-tee and provides strong evidence of correctness