Reasoning for Local Graph over Knowledge Graph with a Multi-Policy Agent

Abstract

The reinforcement learning framework for multi-hop relational paths is one of the effective methods for solving knowledge graph incompletion. However, these models are associated with limited performances attributed to delayed rewards and far-fetched search trajectories. To overcome these challenges, we propose the searching window and multi-policy agent. The searching window provides a large action space, so that the agent can backtrack based on the newly obtained information and establish a local graph instead of a path chain. Based on the searching window, a double long short-term memory (DBL-LSTM) policy network is introduced to encode the local graph and relation sequence, after which the encoding information is used by the agent to select a correct entity to grow the local graph. Furthermore, multi-policy agent separately infers the local graph through three different policy networks, then, all local graphs are integrated into an information-rich local graph. Experiments using the WN18RR dataset revealed that local graph reasoning with searching window had greater rewards than path reasoning, the proposed DBL-LSTM policy network improved all HITS@N(N = 1,3,5,10) compared to prior works, and that the multi-policy agent achieved higher hit rates than single-policy agent.