Multimodal Surface Sensing based on Hybrid Flexible Triboelectric and Piezoresistive Sensor

Abstract

Sensing the surface properties through touch, as the most natural perceptual way of humans, has become an important and practical method for human-machine interactions (HMI) and robot manipulations. In this paper, we design a fingertip hybrid flexible tactile sensor for multimodal surface sensing, based on the triboelectric and piezoresistive mechanisms. A real-time tactile sensing system is implemented on a 3D-printed robot finger together with a wireless data acquisition board. A virtual data generation method is proposed to expand the model adaptability under different compression force levels. Moreover, considering the characteristics of data generated by our sensors, a novel deep learning model with a residual structure is developed, named parallel residual convolutional neural network (PR-CNN). Our model outperforms the state-of-the-art models, i.e., Res-CNN, LSTM-FCN and InceptionTime, with over 96% accuracy, on three classification tasks, including textures (13 types), materials (10 types), and combinations of textures and materials (18 types). The proposed system has broad applications in service robots, industrial sorting robots, and HMI.