Realistic Performance Gain Measurements for XML Data Streaming with Meta Data

This report is part of our ongoing project on the optimization of stream processing for XPath queries on XML documents. The fact that XML documents can be large relative to query-processing memory is one of the reasons to favour streaming over in-core processing. But even in streaming mode query processing uses storage and hence time proportional to the depth of sub-documents. With the goal of querying large documents on mobile devices, or very large ones on normal machines, we have designed a scheme whereby exhaustive searching can be traded against streaming performance. Our scheme uses query meta-data which restricts the search to a subset of the document. In an earlier report we have measured the maximal theoretical gains possible with perfect ”chance ” on choosing the meta-data for synthetic documents, and have found them attrac-tive. In this report we verify two further necessary properties of our scheme, namely 1. the maximal theoretical gains are confirmed on a realistic data set 2. randomly-chosen meta-data also leads to substantial performance gains, which we quantify against the percentage of actual solutions found for the query.