Benchmarking XML Database Systems: First Experiences

We recently developed and published a scaleable multi-user benchmark called XMach-1 (XML Data Management benchmark) for evaluating the performance of XML data management systems [1]. To our knowledge it is the first such benchmark. It aims at realistically evaluating the performance of individual systems as well as to allow for a performance comparison between different systems and architectures ranging from native XML data management systems to XML-enabled relational DBMS. Specifying and implementing the benchmark revealed a number of problems which are partly due to the lack of a standardized XML query language, the complexity of the XML format and the relative immaturity of current XML database software. After a brief review of XMach-1 we will discuss our experiences made so far