Benchmarking Performance-Critical Components in a Native XML Database System

Abstract. The rapidly increasing number of XML-related applications indicates a growing need for efficient, dynamic, and native XML sup-port in database management systems (XDBMS). So far, both industry and academia primarily focus on benchmarking of high-level performance figures for a variety of applications, queries, or documents – frequently executed in artificial workload scenarios – and, therefore, may analyze and compare only specific or incidental behavior of the underlying sys-tems. To cover the full XDBMS support, it is mandatory to benchmark performance-critical components bottom-up, thereby removing bottle-necks and optimizing component behavior. In this way, wrong conclu-sions are avoided when new techniques such as tailored XML operators, index types, or storage mappings with unfamiliar performance character-istics are used. As an experience report, we present what we have learned from benchmarking a native XDBMS and recommend certain setups to do it in a systematic and meaningful way.