STABILIZER: Enforcing Predictable and Analyzable Performance

Modern architectures have made program behavior brittle and un-predictable, making software performance highly dependent on its execution environment. Even apparently innocuous changes, such as changing the size of an unused environment variable, can—by altering memory layout and alignment—degrade perfor-mance by 33 % to 300%. This unpredictability makes it difficult for programmers to debug or understand application performance. It also greatly complicates the evaluation of performance optimiza-tions, since slight changes in the execution environment can have a greater impact on performance than a typical optimization. We present STABILIZER, a compiler and runtime system that enforces reliable performance. By comprehensively randomizing the placement of functions, stack frames, and heap objects in mem-ory, STABILIZER provides predictable performance with high prob-ability. Random placement makes bad layouts unlikely, and re-randomization ensures they are short-lived when they do occur. We demonstrate that STABILIZER effectively eliminates measurement bias with minimal overhead, allowing it to be used in deployment to ensure predictable performance. In addition, STABILIZER enables statistically rigorous performance evaluation. We demonstrate its use by testing the effectiveness of standard optimizations used in the LLVM compiler; we find that, across the SPEC CPU2000 and CPU2006 benchmark suites, the effect of the-O3 optimization level is indistinguishable from noise. 1