A Spatial-Temporal Multi-Feature Network (STMF-Net) for Skeleton-Based Construction Worker Action Recognition

Abstract

Globally, monitoring productivity, occupational health, and safety of construction workers has long been a significant concern. To address this issue, there is an urgent need for efficient methods to continuously monitor construction sites and recognize workers’ actions in a timely manner. Recently, advances in electronic technology and pose estimation algorithms have made it easier to obtain skeleton and joint trajectories of human bodies. Deep learning algorithms have emerged as robust and automated tools for extracting and processing 3D skeleton information on construction sites, proving effective for workforce action assessment. However, most previous studies on action recognition have primarily focused on single-stream data, which limited the network’s ability to capture more comprehensive worker action features. Therefore, this research proposes a Spatial-Temporal Multi-Feature Network (STMF-Net) designed to utilize six 3D skeleton-based features to monitor and capture the movements of construction workers, thereby recognizing their actions. The experimental results demonstrate an accuracy of 79.36%. The significance of this work lies in its potential to enhance management models within the construction industry, ultimately improving workers’ health and work efficiency.