Automated Image-to-BIM Using Neural Radiance Fields and Vision-Language Semantic Modeling

Abstract

This study introduces a novel, automated image-to-BIM (Building Information Modeling) workflow designed to generate semantically rich and geometrically useful BIM models directly from RGB images. Conventional scan-to-BIM often relies on specialized, costly, and time-intensive equipment, specifically if LiDAR is used to generate point clouds (PCs). Typical workflows are followed by a separate post-processing step for semantic segmentation recently performed by deep learning models on the generated PCs. Instead, the proposed method integrates vision language object detection (YOLOv8x-World v2) and vision based segmentation (SAM 2.1) with Neural Radiance Fields (NeRF) 3D reconstruction to generate segmented, color-labeled PCs directly from images. The key novelty lies in bypassing post-processing on PCs by embedding semantic information at the pixel level in images, preserving it through reconstruction, and encoding it into the resulting color labeled PC, which allows building elements to be directly identified and geometrically extracted based on color labels. Extracted geometry is serialized into a JSON format and imported into Revit to automate BIM creation for walls, windows, and doors. Experimental validation on BIM models generated from Unmanned Aerial Vehicle (UAV)-based exterior datasets and standard camera-based interior datasets demonstrated high accuracy in detecting windows and doors. Spatial evaluations yielded up to 0.994 precision and 0.992 Intersection over Union (IoU). NeRF and Gaussian Splatting models, Nerfacto, Instant-NGP, and Splatfacto, were assessed. Nerfacto produced the most structured PCs suitable for geometry extraction and Splatfacto achieved the highest image reconstruction quality. The proposed method removes dependency on terrestrial surveying tools and separate segmentation processes on PCs. It provides a low-cost and scalable solution for generating BIM models in aging or undocumented buildings and supports practical applications such as renovation, digital twin, and facility management.