Mono Regression Benchmarking

Regression benchmarking is a methodology for detecting performance changes in software by periodic benchmarking. Detecting performance regressions in particular helps to improve software quality, similarly as regression testing, which however focuses only on software functionality. To achieve an acceptable level of false alarms, regression benchmarking requires a statistically sound planning and evaluation of the benchmarks. This paper presents a statistical method for automated detection of performance changes and shows how this method performs on a prototype of a regression benchmarking suite for Mono, a large open source project which implements the .Net platform