Using Mutation for the Assessment and Optimization of Tests and Properties∗

We are interested in exploring the complementary relation-ship and tradeoffs between testing and property-based anal-ysis with respect to bug detection. In this paper we present an empirical approach to the assessment of testing and property-based analysis tools using metrics to measure the quantity and efficiency of each technique at finding bugs. We have implemented our approach in an assessment component that has been constructed to allow for symmetrical comparison and evaluation of tests versus properties. In addition to assessing test cases and properties we are also interested in using each to optimize the other as well as to develop hybrid quality assurance approaches. We hypothesize that the synergies of using testing and property-based analysis in combination will allow for optimizations in test suites and property sets that are not possible by using both approaches in isolation. Keywords testing, formal analysis, test suite assessment, property as-sessment, empirical software engineering. 1