Effective Benchmarking for RDF Stores Using Synthetic Data

Abstract. RDF stores are showing consistent performance improvements, with benchmarks showing that several are capable of effectively storing and querying over 109 triples. However, detailed information regarding the capabilities of the available systems is limited due to the fact that current benchmarks provide little configurability, and little depth on the strengths and weaknesses of the stores they test. This paper considers the deficiencies of current benchmarks with regards to measuring the performance of RDF stores, and goes on to describe the creation of a new system to run a greater variety of tests using highly configurable synthetically generated datasets. Finally, the benchmark is applied to existing large scale stores, and the results interpreted. This work is intended to inform future RDF store development, and allow application developers to choose a system appropriate to their specific needs