Deep Learning Based Large-Area Contact Sensing for Safe Human–Robot Interaction Using Conformal Kirigami Structure-Enabled Robotic E-Skin

Abstract

Collaborative robots need to work with people in shared spaces interactively, so a robotic e-skin with large-area contact sensing capability is a crucial technology to ensure human safety. However, realizing real-time contact localization and intensity estimation on a robot body with a large area of continuous and complex surfaces is challenging. Herein, a novel large-area conformal Kirigami structure that can be customized for complex geometries and transform small-area planar sensor arrays into large-area curved conformal e-skin is proposed. This sensor network can effectively detect Lamb/guided wave responses generated by transient hard contact. Additionally, a convolutional neural network-based deep learning algorithm is implemented to decode the features of guided wave signals and predict the contact location and energy intensity on the robot surface. With the deep learning-based method, the accuracy of collision localization can reach 2.85 ± 1.90 mm and the prediction error of collision energy can reach 9.8 × 10−4 ± 8.9 × 10−4 J. Demonstrations show that the proposed method can provide real-time on-site contact sensing, providing a promising solution for future intelligent human–robot interaction.