Comparison of unit-level automated test generation tools

Data from projects worldwide show that many software projects fail and most are completed late or over budget. Unit testing is a simple but effective technique to improve software in terms of quality, flexibility, and time-to-market. A key idea of unit testing is that each piece of code needs its own tests and the best person to design those tests is the developer who wrote the software. However, generating tests for each unit by hand is very expensive, possibly pro-hibitively so. Automatic test data generation is essential to support unit testing and as unit testing is achieving more at-tention, developers are using automated unit test data gen-eration tools more often. However, developers have very little information about which tools are effective. This ex-periment compared three well-known public-accessible unit test data generation tools, JCrasher, TestGen4j, and JUB. We applied them to Java classes and evaluated them based on their mutation scores. As a comparison, we created two additional sets of tests for each class. One test set con-tained random values and the other contained values to sat-isfy edge coverage. Results showed that the automatic test data generation tools generated tests with almost the same mutation scores as the random tests.