High throughput indexing for large-scale semantic web data

Distributed RDF data management systems become increas-ingly important with the growth of the Semantic Web. Cur-rently, several such systems have been proposed, however, their indexing methods meet performance bottlenecks either on data loading or querying when processing large amounts of data. In this work, we propose a high throughout in-dex to enable rapid analysis of large datasets. We adopt a hybrid structure to combine the loading speed of similar-size based methods with the execution speed of graph-based approaches, using dynamic data repartitioning over query workloads. We introduce the design and detailed implemen-tation of our method. Experimental results show that the proposed index can indeed vastly improve loading speeds while remaining competitive in terms of performance. There-fore, the method could be considered as a good choice for RDF analysis in large-scale distributed scenarios. 1