Unsafe-Net: YOLO v4 and ConvLSTM based computer vision system for real-time detection of unsafe behaviours in workplace

Abstract

Unsafe behaviour is a leading cause of death or injury in the workplace, including many accidents. Despite regular safety inspections in workplaces, many accidents occur as a result of breaches of occupational health and safety protocols. In these environments, despite efforts to prevent accidents and losses in hazardous environments, human error cannot be completely eliminated. In particular, in computer-based solutions, automated behaviour detection has low accuracy, is very costly, not real-time and requires a lot of time. In this study, we propose Unsafe-Net, a hybrid computer vision approach using deep learning models for real-time classification of unsafe behaviours in workplace. For the Unsafe-Net, a dataset is first specifically created by capturing 39 days of video footage from a factory. Using this dataset, YOLO v4 and ConvLSTM methods are combined for object detection and video understanding to achieve fast and accurate results. In the experimental studies, the classification accuracy of unsafe behaviours using the proposed Unsafe-Net method is 95.81% and the average time for action recognition from videos is 0.14 s. In addition, the Unsafe-Net has increased the real-time detection speed by reducing the average video duration to 1.87 s. In addition, the system is installed in a real-time working environment in the factory and employees are immediately alerted by the system, both audibly and visually, when unsafe behaviour occurs. As a result of the installation of the system in the factory environment, it has been determined that the recurrence rate of unsafe behaviour has been reduced by approximately 75%.