PointMoment: a mixed-moment self-supervised learning approach for 3D Terracotta Warriors

Abstract

The Terracotta Warriors, a hallmark of China’s cultural heritage, frequently exhibit fragmentation and deformation due to natural factors like earthquakes and human activities. Accurate classification and segmentation of their 3D models are essential for effective restoration. However, the irregularity of the fragmented terracotta pieces renders manual annotation time-consuming and labor-inteśnsive. To address this challenge, we propose a self-supervised learning method utilizing high-order mixed moments for 3D point clouds. It employs a high-order mixed moment loss function instead of the traditional contrastive loss function and does not require special techniques like asymmetric network architectures or gradient stopping. Our method involves calculating the high-order mixed moment of feature variables and forcing them to decompose into individual moments, enhancing variable independence and minimizing feature redundancy. Additionally, we integrate a contrastive learning approach to maximize feature invariance across different augmentations of the same point cloud. Experiments demonstrate that our method outperforms previous unsupervised learning techniques in the downstream tasks of 3D point cloud classification and segmentation. Additionally, our method shows strong performance in the specific tasks related to the Terracotta Warriors. We hope this success can pave the way for new avenues in the virtual protection and restoration of cultural relics. Code is available at https://github.com/caoxin918/PointMoment.