Parallel materialization of datalog programs with spark for scalable reasoning

Abstract

As the volume of semantic data increases rapidly,semantic reasoning becomes a very challenging task. Existing scalable reasoners focus on fragments of OWL 2 RL (eg. RDFS,OWL Horst),and cannot support Semantic Web Rules Language (SWRL) rules,which are widely used in real-world knowledge-based applications. As reasoning of OWL 2 RL ontology extended with SWRL rules can be implemented by materialization of Datalog programs,we propose an approach on parallel materialization of Datalog programs with Spark for scalable reasoning. Since existing scalable reasoners aimed for deterministic rule sets,they used rule-specific strategies for translation of rule execution to parallel jobs and performance optimization techniques. Thus,they cannot be easily extended to support application-specific semantics. In this paper,we propose a rule-independent automatic translation strategy,and several optimization techniques including a dynamic data partition model,a duplication removing strategy and a dependency-aware rule scheduling strategy. These techniques can generalize to vast application-specific semantic rules. Finally,we evaluate our approach with both synthetic and real knowledge bases. The experimental results show our implementation is scalable and the reasoning speed is comparable with that of CiChild,the state-of-the-art scalable reasoner for RDFS/OWL Horst semantics using rule-specific optimizations.