Single-image 3D particle reconstruction via generative AI-empowered large vision models

Abstract

This study presents a diffusion-assisted particle reconstruction model (DPRM), a novel framework for reconstructing high-fidelity 3D particle morphology from a single 2D image of granular assemblies. DPRM leverages cascaded large vision models in three stages: (1) segmentation of individual grains via a U-Net-enhanced segment anything model, (2) multi-view synthesis for each particle using denoising diffusion probabilistic models (DDPMs), and (3) 3D geometry approximation via a DDPM-assisted large reconstruction model, generating simulation-ready mesh representations. After mesh decimation and size correction, the outputs are compatible with discrete element modeling and other physics-based simulations. Quantitative validations confirm DPRM's accuracy in predicting particle size and shape distributions. Crucially, the developed method enables zero-shot generation to novel scenarios without extensive retraining, overcoming limitations of prior methods. This work establishes the first end-to-end pipeline for particle-level 3D reconstruction from monocular scene images, enabling the generation of statistically realistic particle shape for physics-based granular simulations in engineering and industry.