Quantifying the impact of requirements volatility on systems engineering effort

2012-07-06Although changes in requirements are expected as part of a system’s development, excessive volatility after the requirements baseline is likely to result in cost overruns and schedule extensions in large complex systems. Furthermore, late changes in requirements may cause significant rework of engineering products and lead to project failure. Changes in requirements should be expected, accounted for and managed within the context of the system of interest. Unfortunately, system developers lack adequate methods and tools to anticipate and manage the impact of volatile requirements, and cost estimating techniques often fail to account for their economic consequences. ❧ This dissertation presents an extension to COSYSMO, a generally-available parametric systems engineering cost model, which incorporates requirements volatility as a predictor of systems engineering effort within COSYSMO’s structure and scope with the aim of improving the model’s cost estimation capabilities. The requirements volatility model extension was developed through a combination of expert judgment gathered through surveys and discussions in six different research workshops and historical data collected from 25 projects. The null hypothesis that the volatility of requirements throughout a system’s life cycle is not a statistically significant predictor of systems engineering effort was rejected in factor of the alternative hypothesis with a P-value of 0.03. A comparison of the estimation accuracy of COSYSMO to that of the model that includes volatility effects shows an improvement in predictive accuracy from 52% to 80% at the PRED (20) level and a reduction in the mean magnitude error (MMRE) from 21% to 16%. In addition, the coefficient of determination between the predictor, systems engineering size adjusted for diseconomies of scale, and the response, actual systems engineering effort improved from an R2 of 0.85 to an R2 of 0.92 when the volatility factor was applied to the model. ❧ In addition to the mathematical model that quantifies the impact of volatility on systems engineering effort; the contributions of the research include a-) a set of project organizational, technical, and contextual factors ranked by subject matter experts in terms of their influence on requirements volatility; b-) the operationalization of the requirements volatility parameter in COSYMO through a 5-point rating scale; c-) a documented set of observations, developed from the literature and the research workshops, that describe the behaviors and effects of requirements volatility throughout the system life cycle