An annotation tool for benchmarking methods for automated construction worker pose estimation and activity analysis

Abstract

The increase in affordability and quality of jobsite cameras provides opportunities to facilitate monitoring of construction worker activities. Research has highlighted the potential of computer vision algorithms to be used for automated activity analysis. However, applying and adapting such techniques to the construction setting and deploying them for use on job sites entails extensive validation against a construction-specific benchmark dataset. Most state-of-the-art computer vision algorithms are strongly supervised and thus such benchmarks require large amounts of per-image ground truth annotations, both to enable the algorithm to learn how to perform its task during training time and for evaluation on a separate portion of this dataset at test-time. We address this need by introducing an annotation tool that enables users to label visual footage of construction workers by positioning both worker instances and individual key points corresponding to the body parts of each worker (e.g., left wrist, right elbow, nose) in the scene. This annotation tool is the first such publicly available tool capable of producing ground truth for pose estimation, tracking and activity analysis methods to the best of our knowledge. We demonstrate the capabilities of our tool by exhaustively annotating key points, identities and activity labels in a dataset of 393 image sequences that depict workers performing various construction-related activities. This preliminary set of annotation tasks demonstrates the ease of use and flexibility of our annotation tool.