Knowledge-Enhanced Scene Context Embedding for Object-Oriented Navigation of Autonomous Robots

Abstract

Object-oriented navigation in unknown environments with only vision as input has been a challenging task for autonomous robots. Introducing semantic knowledge into the model has been proved to be an effective means to improve the suboptimal performance and the generalization of existing end-to-end learning methods. In this paper, we improve object-oriented navigation by proposing a knowledge-enhanced scene context embedding method, which consists of a reasonable knowledge graph and a designed novel 6-D context vector. The developed knowledge graph (named MattKG) is derived from large-scale real-world scenes and contains object-level relationships that are expected to assist robots to understand the environment. The designed novel 6-D context vector replaces traditional pixel-level raw features by embedding observations as scene context. The experimental results on the public dataset AI2-THOR indicate that our method improves both the navigation success rate and efficiency compared with other state-of-the-art models. We also deploy the proposed method on a physical robot and apply it to the real-world environment.