Interactive Incremental Online Learning of Objects Onboard of a Cooperative Autonomous Mobile Robot

Abstract

Detecting objects and referring to them in a dialog is a crucial requirement for robotic systems that cooperate with humans. For this, in an unrestricted natural environment the innate concepts of the robot must be extended and adapted over time. In this paper we describe an autonomous mobile robot system that performs online interactive incremental learning of objects. We argue that this combination strongly contributes to the variation of appearance, context, and labels under which visual concepts are encountered and thus overcomes limitations of existing databases and robotic systems where one or more of these aspects are missing. In the current prototype version, objects are shown to the robot in hand and are learned by a standard classifier on top of pre-trained CNN features. We evaluate the basic feasibility of the current approach on an existing database of hand-held objects, show how it performs online on the robot, and discuss extensions of the system towards life-long learning and data acquisition.