The role of commitment in software process improvement

Abstract Software process improvement (SPI) approaches have been designed to produce changes at many levels, i.e. in the strategies, culture and working practices, of software development. Studies have shown that nearly two thirds of all SPI efforts have failed or fallen short of expectations. It is often stated in SPI-related literature and practice that "commitment" to SPI plays an important part in determining whether an SPI endeavor ultimately becomes a success or a failure. However, it often remains unclear what this concept actually means and how it affects SPI. This thesis argues for a scientifically grounded concept of commitment and delivers a description and a definition of this concept in the context of software process improvement. The elaboration of the concept is based on a literature study, which makes the research done in behavioral psychology and organizational science applicable in the field of software process improvement. This thesis shows that current thinking relies on practical models of commitment, and the empirically validated analysis conducted within this study reveals a number of common misleading assumptions regarding the notion and development of commitment in SPI. On this basis, this thesis suggests that the commitment phenomenon is better explained through what can be called strategic, operational and personal commitment nets. This framework can be used for analyzing the unfolding and alteration of commitment towards a specific entity, in this case a software process improvement endeavor, through time and changing circumstances. The viability and usefulness of the commitment nets framework is demonstrated through an analysis of four SPI cases in two software organizations. As a result, it is shown that even though the objective features of SPI in terms of costs and benefits may be dominating in the project initiation phase, their role tends to lose strength later on due to an inability of the SPI effort to produce quick and meaningful results, even if these are explicitly sought for. This phenomenon gives rise to a need for enhancing the role of social and psychological drivers. If this is not achieved, SPI activities are likely to cease to exist. The empirical analysis demonstrates that the use of the commitment nets model enables a more precise analysis of the various aspects involved in the commitment phenomenon than what would have been possible with current commitment models. Commitment, as conceptualized and operationalized in this thesis, makes a significant contribution to the outcome of the SPI initiative. The empirical evidence shows that, eventually, even well-planned SPI initiatives may fail to reach the goals set for them due to changes in commitment nets