Hybrid encoding fringe and simulation-to-real scene approach for accurate depth estimation in fringe projection profilometry

Abstract

Fringe projection profilometry (FPP) is a widely adopted technique for three-dimensional reconstruction, and end-to-end depth estimation based on deep learning has gained considerable attention in recent years. However, this approach often suffers from accuracy limitations and typically requires large amounts of real-world data for training. In this study, we propose what we believe to be a novel hybrid-encoded fringe pattern to replace traditional periodic fringes, thereby enhancing depth estimation accuracy. Additionally, we utilize simulated data for training instead of real-scene data and employ the wrapped phase as network input to bridge the domain gap between simulated and real-world scenarios. We introduce a new network architecture, MSAUNet, designed to extract and fuse multi-scale features effectively. The proposed method is evaluated on the largest real-world dataset to date. Experimental results demonstrate that our method outperforms four existing end-to-end depth estimation techniques. Specifically, for an FPP system with a measurement depth range of 120 mm, we achieve a mean absolute error (MAE) of 0.207 mm for the task of simulation-based training and real-scene inference. The source code and dataset will be made publicly available.