Characterization of the Xen project code review process: an experience report

Many software development projects have introduced mandatory code review for every change to the code. This means that the project needs to devote a significant effort to review all proposed changes, and that their merging into the code base may get considerably delayed. Therefore, all those projects need to understand how code review is working, and the delays it is causing in time to merge. This is the case in the Xen project, which performs peer review using mailing lists. During the first half of 2015, some people in the project observed a large and sustained increase in the number of messages related to code review, which had started some years before. This observation led to concerns on whether the code review process was having some trouble, and too large an impact on the overall development process. Those concerns were addressed with a quantitative study, which is presented in this paper. Based on the information in code review messages, some metrics were defined to infer delays imposed by code review. The study produced quantitative data suitable for informed discussion, which the project is using to understand its code review process, and to take decisions to improve it.Preprint of the paper published in the Proceedings of the 13th Working Conference on Mining Software Repositories (MSR 2016), Austin (Texas, USA), May 13--14, 2016, DOI 10.1145/2901739.2901778. The work presented in this paper has been funded in part by the European Union's Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie grant agreement No 642954, and by the Spanish Government, under a Torres Quevedo grant, and through project TIN2014-59400-R