QUERY2BERT: Combining Knowledge Graph and Language Model for Reasoning on Logical Queries

Abstract

Answering logical questions with a knowledge graph has been a critical research focus because this needs to reason and synthesize information. Previous studies have mainly dealt with logical operations using graph embedding techniques, such as conjunctions, disjunctions, and negation. However, these studies have neither effectively organized the data to retrieve multi-hop reasoning quickly nor combined text description to enhance logical operations' semantics. Our study introduces a model called QUERY2BERT, which solves two of the above limitations. Specifically, QUERY2BERT first combined the node2vec and the BERT models to embed a knowledge graph with description information of every entity. Then, embedded nodes were indexed with a K-D tree structure. Finally, we used nearest neighbor search on K-D tree to retrieve neighbor-embedded nodes and implemented logical operations like projection, intersection, union, and negation to find answers to complex questions. We tested our model on three benchmark knowledge graph datasets and showed that QUERY2BERT significantly improved accuracy and speed compared to other state-of-the-art models.