Change classification and its applications to program development and testing

During program development, testing and code editing are interleaved activities. When tests unexpectedly fail, the changes that caused the failure are not always easy to find. We present several analyses that automatically classify changes as Red, Yellow, or Green, indicating the likelihood that they contributed to a test’s failure. We implemented these techniques as an extension of the JUnit testing framework, and evaluated their effectiveness in two case studies. Our results indicate that change classification can effectively focus programmer attention on failure-inducing changes, improving on current manual searching/debugging techniques. Change classification can also determine untested changes, to inform programmers that additional tests are needed. Furthermore, change classification can determine those changes that can be committed safely to a version control repository without breaking any tests, even if a developer’s local workspace contains failing tests. Early adoption of edited code avoids inefficient parallel implementations of the same functionality and possible conflicts when merging independent changes later in the development process.Technical report DCS-TR-57