IOS Press Observing the Web of Data through Efficient Distributed SPARQL Queries

Abstract. Dealing with heterogeneity is one of the key challenges of Big Data analytics. The emergence of Linked Data pro-vides better interoperability and thus further enhances potential of Big Data analytics. The use of Linked Data for analytics raises performance challenges when considering the distribution of data sources and the performance of Linked Data stores in comparison to other storage technologies. This paper investigates the performance of distributed SPARQL approaches to gain an understanding of how well the distribution challenge can be addressed in such an environment. We describe the distributed SPARQL query infrastructure that has been deployed in the Southampton University Web Observatory (SUWO), to support analytics that requires prompt access to a large number of Linked Data datasets. This distributed SPARQL approach adopts dynamic optimisation techniques to take advantage of runtime statistics, and exploits parallelism. The infrastructure is evaluated on Twitter data hosted in the SUWO and popular queries on these data. It demonstrates that the infrastructure described in this paper has a performance advantage over other existing distributed SPARQL engines