Autonomous Docking of Mobile Robots by Reinforcement Learning Tackling the Sparse Reward Problem

Abstract

Most mobile robots are powered by batteries, which must be charged before their level become too low to continue providing services. This paper contributes a novel method based on Reinforcement Learning (RL) for the autonomous docking of mobile robots at their charging stations. Our proposal considers a RL network that is fed with images to visually sense the environment and with distance measurements to safely avoid obstacles, and produces motion commands to be executed by the robot. Additionally, since the autonomous docking is in essence a sparse reward task (the only state that returns a positive reward is when the robot docks at the charging station), we propose the usage of reward shaping to successfully learn to dock. For that we have designed extrinsic rewards that are built on the results of a Convolutional Neural Network in charge of detecting the pattern typically used to visually identify charging stations. The experiments carried out support our design decisions and validate the method implementation, reporting a ∼ 100% of success in the docking task with obstacle-free paths, and ∼ 93% when obstacles are considered, along with short execution times (10 s and 14 s on average, respectively).