Matching Test Cases for Effective Fault Localization

Finding the cause of a program’s failure from a causal-analysis perspective requires, for each statement, tests that cover the statement and tests that do not cover the statement. However, in practice the composition of test suites can be detrimental to effective fault localization for two reasons: (1) lack-of-balance, which occurs if the coverage characteristics of tests that cover a statement differ from tests that do not cover the statement, and (2) lack-of-overlap, which occurs if test cases that reach the control-dependence predecessor of a statement cover or do not cover the statement. This paper addresses these two problems. First, the paper presents empirical results that show that, for effective fault localization, the composition of test suites should exhibit balance and overlap. Second, the paper presents new techniques to overcome these problems—matching to address lack-of-balance and causal-effect imputation to overcome lack-of-overlap—and presents empirical evidence that these techniques increase the effectiveness of fault localization