Angels and Monsters: An Empirical Investigation of Potential Test Effectiveness and Efficiency Improvement from Strongly Subsuming Higher Order Mutation

We study the simultaneous test effectiveness and efficiency improvement achievable by Strongly Subsuming Higher Or-der Mutants (SSHOMs), constructed from 15,792 first order mutants in four Java programs. Using SSHOMs in place of the first order mutants they subsume yielded a 35%-45% reduction in the number of mutants required, while simulta-neously improving test efficiency by 15 % and effectiveness by between 5.6 % and 12%. Trivial first order faults often com-bine to form exceptionally non-trivial higher order faults; apparently innocuous angels can combine to breed monsters. Nevertheless, these same monsters can be recruited to im-prove automated test effectiveness and efficiency. 1