Bolt Loosening Detection Using Key-Point Detection Enhanced by Synthetic Datasets

Abstract

Machine vision based on deep learning is gaining more and more applications in structural health monitoring (SHM) due to the rich information that can be achieved in the images. Bolts are widely used in the connection of steel structures, and their loosening can compromise the safety of steel structures and lead to serious accidents. Therefore, this paper proposes a method for the automatic detection of the bolt loosening angle based on the latest key point detection technology using machine vision and deep learning. First, we built a virtual laboratory in Unreal Engine5 that could automatically label and generate synthetic datasets, and the datasets with bolts were collected. Second, the datasets were trained using the YOLOv7-pose framework, and the resulting model was able to accurately detect key points of bolts in images obtained under different angles and lighting conditions. Third, a bolt loosening angle calculation method was proposed according to the detected key points and the position relationship between neighboring bolts. Our results demonstrate that the proposed method is effective at detecting the bolt loosening angle and that the use of synthetic datasets significantly improves the efficiency of datasets establishment while also improving the performance of model training.