Deep Learning-Assisted Electronic Skin System Capable of Capturing Spatiotemporal and Mechanical Features of Social Touch to Enhance Human–Robot Emotion Recognition

Abstract

In human interactions, social touch communication is widely used to convey emotions, emphasizing its critical role in advancing human–robot interactions by enabling robots to understand and respond to human emotions, thereby significantly enhancing their service capabilities. However, the challenge is to dynamically capture social touch with sufficient spatiotemporal and mechanical resolution for deep haptic data analysis. This study presents a robotic system with flexible electronic skin and a high-frequency signal circuit, utilizing deep neural networks to recognize social touch emotions. The electronic skin, made from double cross-linked ionogels and microstructured arrays, has a low force detection threshold (8 Pa) and a wide perception range (0–150 kPa), enhancing the mechanical resolution of touch signals. By incorporating a high-speed readout circuit capable of capturing spatiotemporal features of social touch gesture information at 30 Hz, the system facilitates precise analysis of touch interactions. A 3D convolutional neural network with a Squeeze-and-Excitation Attention module achieves 87.12% accuracy in recognizing social touch gestures, improving the understanding of emotions conveyed through touch. The effectiveness of the system is validated through interactive demonstrations with robotic dogs and humanoid robots, demonstrating its potential to enhance the emotional intelligence of robots.