MuTEd: A Comparative Study of a Classic and an Extreme Mutation Testing Reporting Tools for Teaching Software Testing

MuTEd, which stands for Mutation Testing Education, is the code name for our master thesis. We conducted a comparative study of two reporting tools for teaching software testing to undergraduates. One, PIT, applies classical mutation testing and the other, Reneri, uses extreme mutation testing. Abstract Although software testing is critical in software engineering, studies have shown a significant gap between students' knowledge of software testing and the industry's needs, hinting at the need to explore novel approaches to teach software testing. Among them, classical mutation testing has already proven to be effective in helping students. We hypothesise that extreme mutation testing could be more effective by introducing more obvious mutants to kill. In order to study this question, we organised an experiment with two undergraduate classes comparing the usage of two tools, one applying classical mutation testing, and the other one applying extreme mutation testing. Unfortunately, the results contradicted our hypothesis, as the students with access to the classic mutation testing tool outperformed the others. Nevertheless, we have developed guidelines based on previous evaluations and our findings