Experiences from Measuring Learning and Performance in Large-Scale Distributed Software Development

Background: Developers and development teams in large-scale software development are often required to learn continuously. Organizations also face the need to train and support new developers and teams on-boarded in ongoing projects. Although learning is associated with performance improvements, experience shows that training and learning does not always result in a better performance or significant improvements might take too long. Aims: In this paper, we report our experiences from establishing an approach to measure learning results and associated performance impact for developers and teams in Ericsson. Method: Experiences reported herein are a part of an exploratory case study of an on-going large-scale distributed project in Ericsson. The data collected for our measurements included archival data and expert knowledge acquired through both unstructured and semi-structured interviews. While performing the measurements, we faced a number of challenges, documented in the form of lessons learned. Results: We aggregated our experience in eight lessons learned related to collection, preparation and analysis of data for further measurement of learning potential and performance in large-scale distributed software development. Conclusions: Measuring learning and performance is a challenging task. Major problems were related to data inconsistencies caused by, among other factors, distributed nature of the project. We believe that the documented experiences shared herein can help other researchers and practitioners to perform similar measurements and overcome the challenges of large-scale distributed software projects, as well as proactively address these challenges when establishing project measurement programs