Many Independent Objective (MIO) Algorithm for Test Suite Generation

Automatically generating test suites is intrinsically a multi- objective problem, as any of the testing targets (e.g, statements to exe- cute or mutants to kill) is an objective on its own. Test suite generation has peculiarities that are quite di erent from other more regular optimi- sation problems. For example, given an existing test suite, one can add more tests to cover the remaining objectives. One would like the smallest number of small tests to cover as many objectives as possible, but that is a secondary goal compared to covering those targets in the rst place. Furthermore, the amount of objectives in software testing can quickly become unmanageable, in the order of (tens/hundreds of) thousands, es- pecially for system testing of industrial size systems. Traditional multi- objective optimisation algorithms can already start to struggle with just four or ve objectives to optimize. To overcome these issues, di erent techniques have been proposed, like for example the Whole Test Suite (WTS) approach and the Many-Objective Sorting Algorithm (MOSA). However, those techniques might not scale well to very large numbers of objectives and limited search budgets (a typical case in system test- ing). In this paper, we propose a novel algorithm, called Many Indepen- dent Objective (MIO) algorithm. This algorithm is designed and tailored based on the speci c properties of test suite generation. An empirical study, on a set of arti cial and actual software, shows that the MIO al- gorithm can achieve higher coverage compared to WTS and MOSA, as it can better exploit the peculiarities of test suite generation