Pooling-Based Feature Extraction and Coarse-to-fine Patch Matching for Optical Flow Estimation

Abstract

This paper presents a pooling-based hierarchical model to extract a dense matching set for optical flow estimation. The proposed model down-samples basic image features (gradient and colour) with min and max pooling, to maintain distinctive visual features from the original resolution to the highly down-sampled layers. Subsequently, patch descriptors are extracted from the pooling results for coarse-to-fine patch matching. In the matching process, the local optimum correspondence of patches is found with a four-step search, and then refined by a velocity propagation algorithm. This paper also presents a method to detect matching outliers by checking the consistency of motion-based and colour-based segmentation. We evaluate the proposed method on two benchmarks, MPI-Sintel and Kitti-2015, using two criteria: the matching accuracy and the accuracy of the resulting optical flow estimation. The results indicate that the proposed method is more efficient, produces more matches than the existing algorithms, and improves significantly the accuracy of optical flow estimation.