Software Testing: Protocol Comparison

: Software testing is hard, expensive, and uncertain. Many protocols have been suggested, especially in the area of conformance verification. In order to compare the efficacy of these protocols, we have implemented a designed simulation experiment that examines performance in terms of testing costs and risks. We find that no single method dominates all others, and provide guidance on how to choose the best protocol for a given application. Software Testing: Protocol Comparison 1 Software Testing Many researchers have proposed protocols for testing software, inspecting software, and deciding when to release software. Practitioners have also developed testing procedures, which often work well but generally lack theoretical justification. Our paper describes a methodology to resolve uncertainty about the relative merits of competing protocols, and presents results that indicate how delicate such comparisons can be. The problem of testing software entails a tradeoff between the cost of ..