Force-Vision Sensor Fusion Improves Learning-Based Approach for Self-Closing Door Pulling

Abstract

Multiple sensors are often used in robotic applications for better situational awareness. Hence, sensor fusion becomes a key technology to manage multiple sources of information and plays a critical role to the success in robotic tasks such as object detection and tracking, autonomous navigation, and interaction with humans. With these capabilities, wheeled autonomous vehicles can be used to automate some public services. However, there are still challenges for wheeled vehicles to safely and agilely maneuvering in human-centered environments. One of these challenges is lacking the capability of autonomously opening doors and traversing doorways without using a general-purpose robotic arm (manipulator). An autonomous door-opening operation is a complex task consisting of identifying the door and door handle, navigating the vehicle to the door, operating the door handle, and pulling or pushing the door to open while traversing the doorway. A self-closing door adds significant difficulty for the last step because the door usually needs to be held open while the vehicle is traversing the doorway. This paper presents a method using force-vision sensor fusion to enhance a deep reinforcement learning (RL) process for a wheeled vehicle to perform the most difficult step of a door-opening and pass-through operation. That step is to pull and hold a self-closing door open while the vehicle is traversing the doorway. In our solution, the vehicle is equipped with a camera, a force sensor, and a concise door-opening mechanism. The method was simulated in Gazebo and the results demonstrated that the deep RL-based force-vision sensor fusion method can be successfully applied to the task of self-closing door pulling by a wheeled vehicle without using a robotic arm and without a pre-planned trajectory. The vehicle control was trained without using domain randomization, but it still works in variant environments.