An Experimental Study of Methods for Executing Test Suites in Memory Constrained Environments

Software for memory constrained mobile devices is often implemented in the Java programming language because the Java compiler and virtual machine (JVM) provide en-hanced safety, portability, and the potential for run-time optimization. However, testing time may increase substan-tially when memory is limited and the JVM employs a com-piler to create native code bodies. This paper furnishes an empirical study that identies the fundamental trade-offs associated with a method that uses adaptive native code un-loading to perform memory constrained testing. The exper-imental results demonstrate that code unloading can reduce testing time by 17 % and the code size of the test suite and application under test by 68 % while maintaining the overall size of the JVM. We also nd that the goal of reducing the space overhead of an automated testing technique is often at odds with the objective of decreasing the time required to test. Additional experiments reveal that using a complete record of test suite behavior, in contrast to a sample-based prole, does not enable the code unloader to make decisions that markedly reduce testing time. Finally, we identify test suite and application behaviors that may limit the effective-ness of our method for memory constrained test execution and we suggest ways to mitigate these challenges.