Problems of Accuracy in the Prediction of Software Quality from Directed Tests.

When software quality is to be evaluated from test data, the evaluation method of choice is usually to construct a reliability model. These models use empirical data and assumptions about the software development process, and they usually result in estimates of the software's proneness to failure. But another paradigm --- one that seeks to establish a certain confidence that the software has high quality --- also appears occasionally. In this paper, we attempt to give a unifying description of that paradigm, and we discuss its usefulness in software testing. This analysis method, which we call probable approximate correctness analysis after a similar paradigm in machine learning, emphasizes the accuracy of quality estimates. We will discuss its use in two situations where it has not previously been applied: the case where software is being developed or maintained, and the case where software testing is not representative of the environment where the program will eventually be used. Our..