Model for Agile Software Development Performance Monitoring

Agile methodologies have been in use for more than ten years and during this time they proved to be efficient, even though number of empirical research is scarce, especially regarding agile software development performance monitoring. The most popular agile framework Scrum is using only one measure of performance: the amount of work remaining for implementation of User Story from the Product Backlog or for implementation of Task from the Sprint Backlog. In time the need for additional measures was acknowledged. According to [Sch05], the stakeholders lack the metrics regarding the finishing project date and better measurement of value in comparison with costs [Yap06]. [Har06] states that the agility principles should be taken into account when adding new measures or the effect can be the opposite [Har06]. The purpose of Doctoral Dissertation was to develop the model for agile software development performance monitoring in compliance with agility principles. Theoretical research was performed to summarize research and practitioners literature in the areas of software development, project management, agile software development and Scrum framework. In the area of software development the following standards have been analyzed: SWEBOK, ISO/IEC 12207: “Information technology – Software life cycle processes”, Capability Maturity Model Integration (CMMI) and COBIT framework from the software development aspect. In the area of project management the following standards have been analyzed: PMBOK and COBIT framework from the project management aspect. In the area of agile software development a definition of agile software development has been presented, followed by comparison between agile and waterfall methodologies, the Scrum framework presentation and the Scrum role description. Further theoretical research has been performed regarding performance monitoring in the areas of agile methods and their maturity, metrics definition, progress reporting and performance monitoring models. The results of this research confirmed the lack of agile software development performance monitoring model that would systematically enable agile project monitoring while using traditional metrics, such as earned value and agile metrics, such as Velocity, while providing simple and transparent reporting and complying with agile principles for collecting and presenting data. Due to its process orientation the concept of PPMS (Process Performance Measurement System) [Kue00] has been selected for the framework for the model development process. The PPMS defines process performance as a degree of stakeholder satisfaction of all stakeholders and process improvement. The resulting AGIT model includes the research results from the Software Technology Laboratory at the Faculty of Computer and Information Science at the University of Ljubljana since year 2005, when they first used Scrum framework for the application software development called “e-student”. Working versions of the model have been used in the academic environment since year 2008 in a capstone course on agile software development using Scrum. The AGIT model has been used in the industrial environment for the first time in year 2011. Lessons learned have been included in the current version of the model as described in the Doctoral Dissertation. The AGIT model will be further developed in the Laboratory. The key science contribution of the Doctoral Dissertation is development of comprehensive AGIT model for agile software development performance monitoring. The research contributions of the AGIT model are: • definition of key performance indicators; • definition of metrics for key performance indicators; • definition of data gathering procedures; • definition of data organization and • definition of reporting. The AGIT model has been developed following the PPS concept. Firstly the software development process using Scrum framework has been described. Then the stakeholders have been determined: Management, Product Owner, Scrum Master and Development Team. For each stakeholder the goals were defined and for each of the goals key performance indicators have been determined. After that the detailed description of key performance indicators have been presented followed by target and limit values for each key performance indicator. Then the data gathering procedures were defined in compliance with agile principles and the data model was presented. In the end the reporting was introduced. The feasibility of the AGIT model was tested using the case study. The basic information about the company and the project studied is presented as well as the subset of the AGIT model key performance indicators used for testing and their graphical presentations: Velocity diagram, Release Burndown diagram, Iteration Burndown diagram, SPI diagram, CPI diagram and BRE spreadsheets. The case study is presented from the time, organizational and quality aspect. Release and Sprint flow are presented followed by the project results. Then the measurement results are presented on the Release level and Sprint level, followed by the results of accuracy analysis of User Story points estimation. The results of case study confirmed that the AGIT model is compliant with the agile principles regarding data collection and data reporting. In the conclusion of Doctoral Dissertation the suggestions for further research are given, based on the results of the case study