An Exploratory Study on Regression Vulnerabilities

Background: Security regressions are vulnerabilities introduced in a previously unaffected software system. They often happen as a result of source code changes (e.g., a bug fix) and can have severe effects. Aims: To increase the understanding of security regressions. This is an important step in developing secure software engineering. Method: We perform an exploratory, mixed-method case study of Mozilla. First, we analyze 78 regression vulnerabilities and 72 bug reports where a bug fix introduced a regression vulnerability at Mozilla. We investigate how developers interact in these bug reports, how they perform the changes, and under what conditions they introduce regression vulnerabilities. Second, we conduct five semi-structured interviews with as many Mozilla developers involved in the vulnerability-inducing bug fixes. Results: Software security is not discussed during bug fixes. Developers' main concerns are the complexity of the bug at hand and the community pressure to fix it. Moreover, developers do not to worry about regression vulnerabilities and assume tools will detect them. Indeed, dynamic analysis tools helped finding around 30% of regression vulnerabilities at Mozilla. Conclusions: These results provide evidence that, although tool support helps identify regression vulnerabilities, it may not be enough to ensure security during bug fixes. Furthermore, our results call for further work on the security tooling support and how to integrate them during bug fixes. Data and materials: https://doi.org/10.5281/zenodo.6792317Comment: This paper has been accepted at ESEM 2022 (16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement