A Coverage Analysis of Java Benchmark Suites

The Java programming language provides an almost ideal environment for both static and dynamic analysis, being easy to parse, and supporting a standardised, easily-profiled virtual environment. In this paper we study the relationship between results obtainable from static and dynamic analysis of Java programs, and in particular the difficulties of correlating static and dynamic results. As a foundation for this study, we focus on various criteria related to run-time code coverage, as commonly used in test suite analysis. We have implemented a dynamic coverage analysis tool for Java programs, and we use it to evaluate several standard Java benchmark suites using line, instruction and branch coverage criteria. We present data indicating a considerable variance in static and dynamic analysis results between these suites, and even between programs in these suites