Recognition method of the main object of three-dimensional photogrammetric modeling of cultural relics

Abstract

Photogrammetry technology helps us reconstruct three dimensional models of cultural relics just by taking photos. However, the background where the cultural relics are located also participates in modeling simultaneously, which wastes storage space and computing resources. Meanwhile, the independence and aesthetics of the three dimensional models are destroyed. Additionally, pure models of cultural relics are obtained by manually deleting the background in three dimensional scenes, which is time consuming and cannot satisfy the practical needs of the flourishing development of digital cultural heritage.This research aims to obtain the three dimensional pure cultural relic models by deleting the redundant background of the photogrammetric model on the basis of object recognition without manual interaction.This paper proposed a method to delete the background of the three dimensional photogrammetric model of cultural relics by objects recognition. First, we recognized the foreground of the cultural relic image by using the deep learning network Mask R-CNN and One Cut, respectively. Second, we extracted the masks of cultural relics by combining the results of Mask R-CNN and One Cut. Last, we applied the masks of cultural relics to delete the background of three dimensional cultural relic models on the basis of the mapping relationships between images and three dimensional models. Moreover, we used the multi-view constraints to optimize the three dimensional recognition accuracy. Additionally, we improved the One Cut method by automatically setting the initial value. In the processing of three dimensional projecting to two dimensional, regarding the cases where triangles overlap, we applied the depth information to distinguish the triangles of foreground and background in three dimensional models.To evaluate proposed method, two cultural relics were selected for the experiments, including Buddha statues in the Beilin Museum in Shaanxi and Mayan masks in the Mexican Museum. We took photos of them and obtained three dimensional models via GET3D (get3d. cn). Our method performs effectively for the Buddha model and the Mayan masks model. Apparently, most of the background of the models is eliminated, and the main bodies of the models are completely preserved. Compared with the artificially labeled ground truth, it can be found that 1) our method preserved three dimensional models complete with a satisfactory recall of 99.23% and 99.20% for the Buddha model and the Mayan masks model, respectively; 2) the algorithm erased the triangles of background with a simplification rate of 85.34% and 86.44% for the Buddha model and the Mayan masks model, respectively; 3) with the advantage of the multi-view constraints, the recognition accuracy of the three dimensional model is higher than two dimensional image.The method proposed in this paper can automatically delete the background of the three dimensional photogrammetric model without manual intervention and preserve the integrity of the object well. The experimental results demonstrate the proposed method is feasible and effective. However, when applied to large three dimensional models, our method is limited to efficiency, given that we distinguished the overlapped triangles successively. Moreover, our pipeline provides a reference for recognizing three dimensional objects in various three dimensional scenes.