The analysis of test coverage using mutation operator BSR

Pokritost programske kode s testi je mera, ki pove, kolikšen delež izvorne kode izbranega programa oziroma poti skozi program je bilo testiranih pri izvajanju testne zbirke. V diplomskem delu smo razvili javansko orodje za analizo pokritosti kode s principi mutacijskega testiranja. Mutacije so majhne modifikacije izvorne kode, ki jih namerno povzročamo v testirani programski kodi z namenom ugotavljanja kvalitete obstoječih testov ter njihove izboljšave. Uporabili smo mutacijski operator BSR ter v javansko izvorno kodo sistematično vstavljali tako imenovane bombe, javanske stavke, ki pri izvajanju povzročijo napako. Na mutirani programski kodi smo poganjali testno zbirko in spremljali, ali se testi uspešno izvedejo. Na osnovi tega smo sklepali na pokritost mesta v kodi, kamor je bila postavljena bomba. Pri tem smo veliko pozornosti posvetili optimizaciji algoritma, ker je mutacijsko testiranje že v osnovi zelo časovno zahteven proces. Aplikacijo smo razvili kot vtičnik za Maven, orodje za upravljanje javanskih projektov, kar omogoča enostavno uporabo na poljubnem projektu. Za cilj smo si postavili razviti orodje, ki je po funkcionalnosti in načinu merjenja pokritosti podobno orodju EMMA, enemu od javanskih orodij za merjenje pokritosti. Pri testiranju aplikacije na nekaj realnih projektih se je pokazalo, da so rezultati precej natančni in primerljivi z orodjem EMMA.Code coverage is a measure used to describe the degree to which the source code of a program or paths through the program has been tested by a particular test suite. In this thesis we have designed and implemented a Java tool for code coverage analysis which is based on principles of mutation testing. Mutations are small modifications of source code which are deliberately seeded into the source code under testing in order to evaluate the quality of existing test suite and improve it. We used the mutation operator BSR to systematically seed so-called bombs into Java source code. In mutation testing a bomb is a statement which causes an exception when executed. After that we ran the test suite against the mutated code and checked whether it finished successfully. Based on the test suite results it is possible to assess the code coverage at the position where the bomb was placed. We put a lot of effort in algorithm optimization because mutation testing is known to be extremely time-consuming process. We designed the application as a plugin for Maven, software project management and comprehension tool, which makes it simple to use on any Maven based Java project. Our goal was to develop a tool similar to EMMA by functionality and code coverage measurement methodology. By running the code coverage analysis on a few real-world open source projects, the application proved to be pretty accurate and comparable to EMMA