An industrial case study of automatically identifying performance regressioncauses

Even the addition of a single extra field or control statement in the source code of a large-scale software system can lead to performance regressions. Such regressions can considerably degrade the user experience. Working closely with the mem-bers of a performance engineering team, we observe that they face a major challenge in identifying the cause of a per-formance regression given the large number of performance counters (e.g., memory and CPU usage) that must be ana-lyzed. We propose the mining of a regression-causes repos-itory (where the results of performance tests and causes of past regressions are stored) to assist the performance team in identifying the regression-cause of a newly-identified regres-sion. We evaluate our approach on an open-source system, and a commercial system for which the team is responsible. The results show that our approach can accurately (up to 80 % accuracy) identify performance regression-causes using a reasonably small number of historical test runs (sometimes as few as four test runs per regression-cause)