An exploration of software faults and failure behaviour in a large population of programs

A large part of software engineering research suf-fers from a major problem—there are insufficient data to test software hypotheses, or to estimate parameters in models. To obtain statistically significant results, a large set of programs is needed, each set comprising many programs built to the same specification. We have gained access to such a large body of programs (writ-ten in C, C++, Java or Pascal) and in this paper we present the results of an exploratory analysis of around 29,000 C programs written to a common specification. The objectives of this study were to characterise the types of fault that are present in these programs; to characterise how programs are debugged during devel-opment; and to assess the effectiveness of diverse pro-gramming. The findings are discussed, together with the potential limitations on the realism of the findings. 1