Automatic Screening of Bolts with Anti-Loosening Coating Using Grad-CAM and Transfer Learning with Deep Convolutional Neural Networks

Abstract

Most electronic and automotive parts are affixed by bolts. To prevent such bolts from loosening through shock and vibration, anti-loosening coating is applied to their threads. However, during the coating process, various defects can occur. Consequently, as the quality of the anti-loosening coating is critical for the fastening force, bolts are inspected optically and manually. It is difficult, however, to accurately screen coating defects owing to their various shapes and sizes. In this study, we applied deep learning to assess the coating quality of bolts with anti-loosening coating. From the various convolutional neural network (CNN) methods, the VGG16 structure was employed. Furthermore, the gradient-weighted class activation mapping visualization method was used to evaluate the training model; this is because a CNN cannot determine the classification criteria or the defect location, owing to its structure. The results confirmed that external factors influence the classification. We, therefore, applied the region of interest method to classify the bolt thread only, and subsequently, retrained the algorithm. Moreover, to reduce the learning time and improve the model performance, transfer learning and fine tuning were employed. The proposed method for screening coating defects was applied to a screening device equipped with an actual conveyor belt, and the Modbus TCP protocol was used to transmit signals between a programmable logic controller and a personal computer. Using the proposed method, we were able to automatically detect coating defects that were missed by optical sorters.