A knowledge-based task planning approach for robot multi-task manipulation

Abstract

Task planning is a crucial component in facilitating robot multi-task manipulations. Language-based task planning methods offer practicality in receiving commands from humans in real-life scenarios and require only low-cost labeled data. However, existing methods often rely on sequence models for planning, which primarily focus on mapping language to sequences of sub-tasks while neglecting the knowledge about tasks and objects. To overcome these limitations, we propose a knowledge-based task planning approach called Recurrent Graph Convolutional Network (RGCN). It is devised with a novel structure that combined GCN (Kipf and Welling in International Conference on Learning Representations (ICLR), 2017) and LSTM (Hochreiter and chmidhuber in Neural Comput 9 (8): 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735) which enables it to leverage knowledge graph data and historical predictions. The experimental results demonstrate that our approach achieves the impressive task planning success rate of (Formula presented.), surpassing the best baseline method significantly, which achieves (Formula presented.). Furthermore, we evaluate the performance of multi-task manipulation across a specific set of 20 tasks within a simulated environment. Notably, RGCN combined with pre-trained primitive tasks exhibits the highest success rate compared with state-of-art multi-task learning methods. Our method is proven to be significant for language-conditioned task planning and is qualified for instructing robots for multi-task manipulation.