JBrainy: Micro-benchmarking Java Collections with Interference (Work in Progress Paper)

Software developers use collection data structures extensively andare often faced with the task of picking which collection to use.Choosing an inappropriate collection can have major negativeimpact on runtime performance. However, choosing the right collectioncan be difficult since developers are faced with many possibilities,which often appear functionally equivalent. One approachto assist developers in this decision-making process is to microbenchmarkdatastructures in order to provide performance insights. In this paper, we present results from experiments on Java collections(maps, lists, and sets) using our tool JBrainy, which synthesisesmicro-benchmarks with sequences of random method calls.We compare our results to the results of a previous experiment onJava collections that uses a micro-benchmarking approach focusedon single methods. Our results support previous results for lists, inthat we found ArrayList to yield the best running time in 90% ofour benchmarks. For sets, we found LinkedHashSet to yield thebest performance in 78% of the benchmarks. In contrast toprevious results, we found TreeMap and LinkedHashMap to yield betterruntime performance than HashMap in 84% of cases