Is coverage a good measure of testing effectiveness

Abstract. Most approaches to testing use branch coverage to decide on the qual-ity of a given test suite. The intuition is that covering branches relates directly to uncovering faults. The empirical study reported here applied random testing to 14 Eiffel classes for a total of 2520 hours and recorded the number of uncovered faults and the branch coverage over time. For the tested classes, (1) random test-ing reaches 93 % branch coverage (2) it exercises almost the same set of branches every time, (3) it detects different faults from execution to execution, (4) during the first 10 minutes of testing, while branch coverage increases rapidly, there is a strong correlation between branch coverage and the number of uncovered faults, (5) over 50 % of the faults are detected at a time where branch coverage hardly changes, and the correlation between branch coverage and the number of uncov-ered faults is weak. These results provide evidence that branch coverage is not a good stopping cri-terion for random testing. They also show that branch coverage is not a good indicator for the effectiveness of a test suite