Cloth recognition based on fabric properties and tactile sensing

Abstract

Objective: With the development of the textile industry, the manual identification of cloth has been unable to meet the growing demand for production. More and more image recognition technologies are applied to cloth recognition. Image recognition is a technology that combines feature extraction and feature learning; it plays an important role in improving the competitiveness of the clothing industry. Compared with general-purpose images, cloth images usually only show subtle differences in texture and shape. Current clothing recognition algorithms are based on machine learning; that is, they learn the features of clothing images through machine learning and compare the features of known fabric to determine the clothing category. However, these clothing recognition algorithms usually have low recognition rates because they only consider the vision attribute, which cannot fully describe the fabric and ignores the properties of the fabric itself. Touch and vision are two important sensing modalities for humans, and they offer complementary information for sensing cloth. Machine learning can also benefit from such multimodal sensing ability. To solve the problem of low recognition accuracy of common fabrics, a fabric image recognition method based on fabric properties and tactile sensing is proposed. Method: The proposed method involves four steps, including image measurement, tactile sensing, fabric learning, and fabric recognition. The main idea of the method is to use AlexNet to extract tactile image features adaptively and match the fabric properties extracted by MATLAB morphology. First, the geometric measurement method is established to measure the input fabric image samples, and a parametric model is obtained after quantitatively analyzing the three key factors by testing the recovery, stretching, and bending behavior of different real cloth samples. The geometric measures of fabric properties can be obtained through parametric modeling. Second, fabric tactile sensing is measured through tactile sensor settings, and the low-level features of tactile images are extracted using convolutional neural network (CNN). Third, the fabric identification model is trained by matching the fabric geometric measures with the extracted features of tactile image and parameter learning through the CNN to learn the different parameters of fabric properties. Finally, the fabric is recognized, and results are obtained. In this study, the issue on cloth recognition is addressed by the basis of tactile image and vision; in this manner, missing sensory information can be avoided. Furthermore, a new fusion method named deep maximum covariance analysis (DMCA) is utilized to learn a joint latent space for sharing features through vision and tactile sensing, which can match weak paired vision and tactile data. Considering that the current fabric dataset contains only a few fabric types, which cannot be classified as everyday fabric, two fabric sample datasets are constructed. The first is a fabric image dataset for fabric property measurement, including the recovery, stretching, and bending images of 12 kinds of fabric types, such as coarse cotton, fine cotton, and canvas. Each type of fabric has 10 images, thus having a total of 360 images. The second is a fabric tactile image dataset, which includes 12 fabric types, each comprising 500 images with a total of 6 000 images. The size of all images are set to 227 × 227 pixels for the convenience of the experiment. Result: To verify the effectiveness of the proposed method, experiments are performed on 12 common fabric samples. Experimental results show that the recognition average accuracy can reach 89.5%. Compared with the method of using only a single and three kinds of fabric attributes, the proposed method obtains a higher recognition rate. The proposed method also possesses better recognition effect compared with that of the mainstream methods. For example, compared with recognition accuracy of sparse coding (SC) combined with support vector machine (SVM), that of the proposed method increases to 89.5%. Conclusion: A fabric image recognition method of combining vision and tactile sensing is proposed. The method can accurately identify fabric for clothing and improve the accuracy of fabric recognition. For the feature extraction task, the AlexNet network achieves simplified high-dimensional features, which can adaptively extract effective features to avoid manual screening. Moreover, the DMCA model performs well in cross-modal matching. Compared with other clothing recognition methods, our method shows several advantages in terms of accuracy, without the cost of expensive equipment. However, our method does not consider the recognition accuracy problem, which is influenced by a small number of samples, a low image measurement data dimension, and a lack of tactile information. In the future, the issues to improve the recognition accuracy of various fabric types will be focused on further.