Plenoptic camera calibration based on microlens distortion modelling

Abstract

Light field cameras have become a popular device for handheld, passive depth estimation. The geometry of the captured light rays can be equivalently described as a virtual camera array, where each view is taken from a different point on a regular grid. With this paper, we propose describing the effect of the main lens and microlens distortion on both the captured views, and on their position on this grid. It is showed that the microlens distortion affects the grid shape, implying a non-constant baseline between each view, unlike assumed in previous light field geometrical models. Moreover, we propose an end-to-end calibration method based on grid point detection. The detection is performed on raw hexagonal lattice sub-aperture images, to avoid a loss of precision in re-sampling operations. The calibration pipeline is tested on a Lytro Illum camera and is compared to the previous state of the art in this field. It is shown that the quality of the reconstructed 3D is significantly improved with our method, thanks to the fine-level description of the baseline between the virtual views. Our model is resilient to a change in the focusing distance of the main lens, making it applicable in a wide variety of scenarios.