A lightweight neural radiance field model with automatic semantic segmentation for post-earthquake building three-dimensional reconstruction

Abstract

Earthquake can cause severe urban damage and economic losses, making the rapid, accurate, and safe exploration of building damage and preservation of on-site earthquake data critical components. Deep learning-based post-earthquake technologies using images collected by unmanned aerial vehicle (UAV) equipment has emerged as a prominent research focus. Among these technologies, three-dimensional (3D) reconstruction is widely utilized but suffers from high computational load and the inability of automatic recognition. This study presents a 3D reconstruction method with a lightweight network architecture and automatic semantic segmentation for post-earthquake buildings based on the idea of neural radiance fields (NeRF), which is called lightweight NeRF with automatic semantic (LNAs). Specifically, LNAs adopt the multi-resolution hash encoder from instant neural graphics primitives for dense space encoding, thereby reducing the computational overhead inherent in the original NeRF, and a subsequent lightweight three-branch multi-layer perceptron for mapping spatial coordinates into volume density, color, and semantic, which realizes the 3D continuous model with 3D semantic. The above process is rapidly trained with UAV-sampled building images and its true semantics. Experiments demonstrate that LNAs realize significant performance improvements over recent NeRF-based framework, delivering 3D reconstruction with 26.20 peak signal-to-noise ratio, and multiple-class 3D semantic segmentation with 84.52% mean intersection over union in 1.33 h. This paper also experiments the stability of LNAs across various datasets and hyperparameter configurations.