Snow Removal from Light Field Images

Abstract

In this paper, we propose a novel deep convolutional neural network (DCNN) for removing snowflakes from light field (LF) images. We observe that snowflakes in LF images always interrupt slopes in background scenes in epipolar plane images (EPIs), which means that snowflakes may be easily detected in EPIs. Our method takes 3D EPI volumes (i.e., stacked subaperture views along the same row or column of an LF image) as input. In this way, our snowflake detector based on a 3D residual network with a convolutional long short-term memory (ResNet-ConvLSTM) can utilize both contextual information and 3D scene structural information to effectively detect snowflakes of different sizes in LF images. Then, an encoder-decoder-based LF image restoration network is proposed to restore the background image. Finally, extensive experiments for comparison with the state-of-the-art methods demonstrate the effectiveness of our method for challenging scenes.