Construction Scene Parsing (CSP): Structured Annotations of Image Segmentation for Construction Semantic Understanding

Abstract

Images ubiquitously support visualization and documentation of as-built status of built infrastructure. Images captured on-site during construction contain rich semantic information, such as object categories, materials and topological relationships, which are useful for many applications, such as progress monitoring, crack detection, quality control, and safety management. Recent advancements in deep learning and convolutional neural network can effectively extract various types of semantic information from images, but they require annotated datasets for model training. Most existing scene parsing datasets contain annotated images captured at project closeout and there lacks a dataset that can be used for construction site scene understanding. In order to support construction scene understanding, we present the Construction Scene Parsing (CSP), an annotated image dataset that contains over 150 construction scenes with image segmentation labelled by experts. The CSP dataset have two primary contributions: 1) It provides a hierarchical semantic structure rather than a unitary label for each image to deal with incomplete and changing components presented on construction images; 2) It provides pixel-wise annotations for every scene and can support various types of scene understanding tasks, such as object recognition, semantic segmentation, instance segmentation and panoptic segmentation. The dataset can be accessed at https://github.com/yugitw/Construction-Scene-Parsing.