PSLFM: a single-frame uncalibrated photometric stereoscopic light field measurement scheme based on dense convolutional neural networks

Abstract

In the realm of 3D measurement, photometric stereo excels in capturing high-frequency details but suffers from accumulated errors that lead to low-frequency distortions in the reconstructed surface. Conversely, light field (LF) reconstruction provides satisfactory low-frequency geometry but sacrifices spatial resolution, impacting high-frequency detail quality. To tackle these challenges, we propose a photometric stereoscopic light field measurement (PSLFM) scheme that harnesses the strengths of both methods. We have developed an integrated information acquisition system that requires only a single data acquisition and does not necessitate the light source vectors as input. This system enables uncalibrated multispectral photometric stereo reconstruction using a dense convolutional neural network (DCN). After that, the two reconstruction results are processed by frequency domain filtering, and the processed results are fused according to a certain weight, which can be adaptively determined by the algorithm according to the reconstruction error. Utilizing a light field camera as the sole acquisition device allows for natural alignment of data, mitigating registration errors. Our approach demonstrates effectiveness across both online datasets and laboratory samples, achieving an error of about 10° and lower in uncalibrated scenarios, with notable generalization. In conclusion, the proposed method facilitates single-frame measurement without calibration and exhibits strong robustness, which is expected to exert significant influence in the fields of machine vision, 3D printing and manufacturing, as well as virtual reality and augmented reality.