Deep Learning Methods Integration for Improving Natural Interaction Between Humans and an Assistant Mobile Robot in the Context of Autonomous Navigation

Abstract

This paper describes a full navigation architecture which includes a set of available Deep Learning-based modules, focused on “speech to text” and “text to speech” translation, and face recognition, for enabling natural interaction between a smart mobile assistant robot and its human users, in a context of autonomous navigation. The system is novel because it allows complex spoken commands to be syntactically analyzed in Spanish and transformed into motion plans, ready to be executed by the robot, by using the well-known Navigation stack included in the ROS ecosystem. A novel computationally efficient approach (to semantically label the free space from raw data provided by a low cost laser scanner device), enables the generation of a labelled polygonal map, which enhances fixed and mobile obstacle avoidance and local planning in real indoor environments. Robot configuration, learned maps, and Deep Learning models adapted to each scenario are safely and privately stored in Google Cloud, allowing the robot to adapt its behavior to different users and settings. The tests which demonstrate the performance of the system, both in simulation and real environments, are also described.