Material augmented semantic segmentation of point clouds for building elements

Abstract

Point clouds are utilized to enable automated engineering applications for their ability to represent spatial geometry. However, they inherently lack detailed surface textures, posing challenges in differentiating objects at the texture level. Hence, this study introduces a 2D–3D fusing approach, leveraging material properties recognized from registered images as an augmented feature to enhance deep learning methods for the segmentation of building elements within point clouds. The proposed method was evaluated quantitatively on a 3D indoor data set with an implementation in an office room. The results are promising, showing improvement in recognition performance, particularly for objects with similar geometry but having different material properties. For instance, the segmentation of boards increased by 70.87%, and doors improved by 41.06% using the PointNet architecture. This enhanced segmentation not only reduces the time for interpreting point clouds but also has the potential to benefit downstream applications such as Scan-to-building information modeling (BIM), as defining regions for objects is essential.