Robust estimations of fault content with capture-recapture and detection profile estimators

Inspections are widely used in the software engineering community as efficient contributors to reduced fault content and improved product understanding. In order to measure and control the effect and use of inspections, the fault content after an inspection must be estimated. The capture-recapture method, with its origin in biological sciences, is a promising approach for estimation of the remaining fault content in software artefacts. However, a number of empirical studies show that the estimates are neither accurate nor robust. In order to find robust estimates, i.e., estimates with small bias and variations, the adherence to the prerequisites for different estimation models is investigated. The basic hypothesis is that a model should provide better estimates the closer the actual sample distribution is to the model's theoretical distribution. Firstly, a distance measure is evaluated and secondly a X2-based procedure is applied. Thirdly, smoothing algorithms are tried out, e.g., mean and median values of the estimates from a number of estimation models. Based on two different inspection experiments, we conclude that it is not possible to show a correlation between adherence to the model's theoretical distributions and the prediction capabilities of the models. This indicates that there are other factors that affect the estimation capabilities more than the prerequisites. Neither does the investigation point out any specific model to be superior. On the contrary, the Mh-JK model, which has been shown as the best alternative in a prior study, is inferior in this study. The most robust estimations are achieved by the smoothing algorithms