Unbiased 3D Semantic Scene Graph Prediction in Point Cloud Using Deep Learning

Abstract

As a core task of computer vision perception, 3D scene understanding has received widespread attention. However, the current research mainly focuses on the semantic understanding task at the level of entity objects and often neglects the semantic relationships between objects in the scene. This paper proposes a 3D scene graph prediction model based on deep learning methods for scanned point cloud data of indoor scenes to predict the semantic graph about the class of entity objects and their relationships. The model uses a multi-scale pyramidal feature extraction network, MP-DGCNN, to fuse features with the learned category-related unbiased meta-embedding vectors, and the relationship inference of the scene graph uses an ENA-GNN network incorporating node and edge cross-attention; in addition, considering the long-tail distribution effect, a category grouping re-weighting scheme is used in the embedded prior knowledge and loss function. For the 3D scene graph prediction task, experiments on the indoor point cloud 3DSSG dataset show that the model proposed in this paper performs well compared with the latest baseline model, and the prediction effectiveness and accuracy are substantially improved.