A model for the quantitative tracking of the software process compliant with the fourth level of the capability maturity model (CMM)

At this point in time a higher level of quality in software production is necessary for the effective development and long lasting survival of software organizations. A controlled software process is the key issue for quality. The Capability Maturity Model (CMM) is a successful quality model, which defines the characteristics of such a process. This model defines five maturity levels; each containing a number of precisely described key process areas and goals. It can be used as a successful guide for software process improvement in software organizations of different sizes. The CMM is particularly appropriate for software process improvement in small software organizations that develop a specific kind of software, namely information systems. The main characteristics of these organizations are restricted resources (people, money) and a specific software process. Quality remains an important issue for such organizations and its achievement requires the ability to suitably measure the software process. The goal of this dissertation is to define a software measurement model for small software organizations that produce information systems. Our model is consistent with the definition of the fourth maturity level of the CMM. Our solution is built on two principles. The fourth level of the CMM defines the software process and measurement very precisely, making the implementation of a measurement model rather complicated and complex. Yet small organizations expect a simple, understandable, and practical solution to the problem of measurement. Our solution offers a compromise of both principles: it is consistent with all reasonable demands of the CMM in small organizations and provides a practical and usable solution for typical small organizations. The complete solution consists of three parts: a) the software process is consistent with the goals of the fourth maturity level of CMM and expresses software production in a small software organization that develops information systems, b) the measurement process describes how measurement is done and fits a defined software process, and c) the measurement information model defines the content of measurement and supports the measurement process. All three parts compose the model for quantitative tracing of software development. a) The first step is to accurately define the software process. The CMM is a complex quality model and as such it is too complicated for direct use in a small organization. We examine two solutions to this problem. First, it is possible to replace the CMM with one of two well-known process models that implement CMM principles in a simpler way, or second, it is possible to adapt the CMM for use in small organizations. The Personal Software Process (PSP) and the Team Software Process (TSP) are two suitable models for CMM replacement, but they generate an approximate and incomplete process from the point of view of small organizations. In the second solution we define a form of adaptation of the CMM for small organizations. The combination of both solutions is the basis for our definition of the adaptive software process, which is the final result of the first part of dissertation. Three innovations are implemented in the adaptive software process. They enable the process to be defined precisely, and at the same time remain dynamically sensitive to changes during software development. First, the software process is defined as a sequence of seven cycles, each consisting of some phases and many steps. Cycles define manageable parts of software development and enable exact planning and timely reactions. Second, each step can be executed in a more or less complete way. This makes our process extendable. Third, our process defines precisely how to forego certain cycles, phases or steps if they are not needed in the concrete software project. In our dissertation we define and demonstrate the implementation of a completely adaptable software process. In the second part of the dissertation we define the measurement approach, which is integrated in the adaptive software process already described. The measurement approach is based on an adequately defined and well-known approach named the Practical Software Measurement (PSM), which complies with modern measurement standards. The PSM consists of two models: the Measurement Process Model and the Measurement Information Model. We adapt both for use in small organizations. b) The Measurement Process Model defines four main measurement activities: how to establish and sustain measurement commitment, how to plan measurement, how to perform it, and how to evaluate measurement. In our approach we divide all measurement activities into two parts: the permanent part defines obligatory measurement issues for each project in a small organization, while the special part complies with the special needs of a particular project. The performer of each project development activity has to collect a defined set of measurement data. The measurement results are stored in an automated central repository where some other measurement results are also calculated. In this way we collect low-level measurement data enabling a more accurate measurement and a simplified operational task for the measurement analyst. However, we demand the disciplined operation of each project member in the sense of PSP, even while the measurement activities are executed in a simple and always identical manner. The analysis of measurement data defines the remaining core measurement activities. We define four types of analysis: assessment, feasibility analysis, performance analysis, and prediction. The first and last types are the most significant in our measurement approach. Assessment is a crucial type of analysis for punctual planning and this is the reason for its special role in the adaptive software process. In this part we implement some innovations that make assessments more exact. All assessments use statistical methods and are based on data regarding similar past projects. Prediction is our genuine type of analysis enabling the measurement analyst to dynamically follow any changes in a project’s characteristics. It defines a constant set of measurement indicators that present the main relations between current measurement data, planned values, and past project data. We define a fully automated mode of prediction in which indicators dynamically change in correspondence with changes in the current measurement data. Automated prediction can replace other types of analysis that are executed only on demand. All the analysis components of prediction in this model are automated and built into the central repository information system. c) The last part of the dissertation presents the content of measurement, which is consistent for all the described solutions. The content description has four levels: • the Measurement Information Model defines the structure of the content description, • the Meta model describes the characteristics of all levels of measurement data and relations between them, • collections of measurement entities, attributes, measures, indicators and measurement issues define measurement precisely, and • the measurement repository implements the described measurement. In this part we present an original method for the definition of a suitable collection of measures and indicators from a defined set of measurement issues. Our combined method is based on the information needs of a small software organization, but it always measures only a well-defined set of attributes of existing entities. As such it unites the best characteristics of both top-down and bottom-up methods. Collections of measurement entities, attributes, (basic, composed and calculated) measures, indicators, and measurement issues are the tangible results of our dissertation. They were all defined using our combined method and they are consistent with the adaptive software process. The collection of permanent measurement issues is derived from the goals of the fourth level of the CMM, as well as from the characteristics of a small organization. However, measurement entities (objects of measurement) originate in the adaptive software process. A collection of basic measures directly measures the main characteristics of entities: the size of products, time consummation, progress, resources, costs, and product quality. A collection of composed measures defines ways to unite similar measurement results in new results, while calculated measures describe how to calculate new results based on those already known. A collection of indicators is the main result of the use of the combined method. It represents the connection between measures and measurement issues. Alternatively, indicators are results of the form of analysis we defined in the measurement process. All three parts of the dissertation define a complete model for the quantitative tracing of software development in a small organization that produces information systems. The model we have created is compliant with the fourth level of the CMM. Our goal was the use of clear concepts, an adaptive specialised definition, and above all practical application. Our model has been presented to several small software organizations in Slovenia. After the presentation the organization representatives were asked to complete a questionnaire. The results of the questionnaire show that our model is in general appropriate and effective for small software organizations. This is especially true if we take into account the different levels of maturity, diverse goals, and various experiences in the field of software measurement of each organization