eRDF: A scalable framework for querying the Web of Data

Abstract

As the Web of Data increases in data size, speed of updates, and heterogeneity, new mechanisms are needed in order to query it effectively. In this paper, we detail eRDF: a novel framework based on evolutionary methods that enables queries over the live Web of Data. We define the problem of finding approximate solutions for queries on the Web of Data as a constrained optimisation problem. Based on this problem definition, we discuss the eRDF framework and a corresponding reference implementation. We present the results of an extensive evaluation addressing the issues of query complexity, result quality, and scale. Key results include: the capability to answer queries over 78 live endpoints containing roughly 14.7 billion triples with minimal computing resources (i.e. a laptop); the capability to process complex SPARQL queries comprising 500 or more triple patterns; and the capability to give some answers to SPARQL queries over DBPedia that currently cannot be processed by that endpoint. Keywords: federated query, approximation, sparql, triple store, evolutionary algorithms