Learning depth from monocular sequence with convolutional LSTM network

Abstract

Resolving depth from monocular RGB image has been a long-standing task in computer vision and robotics. Recently, deep learning based methods has become a popular algorithm on depth estimation. Most existing learning based methods take image-pair as input and utilize feature matching across frames to resolve depth. However, two-frame methods require sufficient and static camera motion to reach optimal performance, while camera motion is usually uncontrollable in most application scenarios. In this paper we propose a recurrent neural network based depth estimation network. With the ability of taking multiple images as input, recurrent neural network will decide by itself which image to reference during estimation. We train a u-net like network architecture which utilizes convolutional LSTM in the encoder. We demonstrate our proposed method with the TUM RGB-D dataset, where our proposed method shows the ability of estimating depth with various sequence lengths as input.