A Texture Integrated Deep Neural Network for Semantic Segmentation of Urban Meshes

Abstract

3-D geo-information is essential for many urban related applications. Point cloud and mesh are two common representations of the 3-D urban surface. Compared to point cloud data, mesh possesses indispensable advantages, such as high-resolution image texture and sharp geometry representation. Semantic segmentation, as an important way to obtain 3-D geo-information, however, is mainly performed on the point cloud data. Due to the complex geometry representation and lack of efficient utilizing of image texture information, the semantic segmentation of the mesh is still a challenging task for urban 3-D geo-information acquisition. In this article, we propose a texture and geometry integrated deep learning method for the mesh segmentation task. A novel texture convolution module is introduced to capture image texture features. The texture features are concatenate with nontexture features on a point cloud that represents by the center of gravity (COG) of the mesh triangles. A hierarchical deep network is employed to segment the COG point cloud. Our experimental results show that the proposed network significantly improves the accuracy with the introduced texture convolution module (1.9% for overall accuracy and 4.0% for average F1 score). It also compares with other state-of-the-art methods on the public SUM-Helsinki dataset and achieves considerable results.