Balancing Time-to-Market and Quality in Embedded Systems

Finding a balance between the time-to-market and quality of a delivered product is a daunting task. The optimal release moment is not easily found. We propose to use historical project data to monitor the progress of running projects. From the data we inferred a formula providing a rough indication of the number of defects given the effort spent thus far (effort-to-defect formula). Furthermore, we provide a worst case bound to the allowed number of residual defects at the end of a project in order to achieve the required level of quality. For this purpose we slightly modified a well-known reliability growth model by Bishop and Bloomfield. It turned out that the software in Philips' MRI scanners has a defect rate of 1 per 1175 device-years of observation. This coincides with the second highest safety integrity level (SIL3) as defined in the IEC 61508 standard. The highest level (SIL4) is only attainable by applying redundancy. Finally, we combine the effort-to-defect formula with the reliability growth model to monitor the progress of a project and to determine when the required level of quality will be reached. We show that a common fault distribution, the Rayleigh model, is not necessarily the best model for predicting the number of residual defects in the system. Using a well-known data analysis approach called exploratory data analysis (EDA) we obtained an alternative model based on the Normal curve. We have evaluated the Rayleigh model and our model based on the Normal curve at Philips Healthcare MRI. The Normal curve predicts defects over time better than the Rayleigh model in the case of Philips Healthcare MRI. Furthermore, time series models (ARIMA) are also useful for accurately describing the defect trend, but are not suitable for long-term predictions. Finally, cost estimation models (COCOMO) lack the predictive capabilities of models fitted on the data using EDA. Their capabilities are limited compared to a model derived from data which reflects the constitutional knowledge of the actual realization of systems within a specific company. However, they can still be used to advantage when there is limited or no data available to use as a basis for lifting from gut feel to order of magnitude. © 2013 Wiley Periodicals, Inc