Sparse depth calculation using real-time key-point: Detection and structure from motion for advanced driver assist systems

Abstract

This paper presents a system for calculating depth using a single camera with a focus on advanced driver assist systems. The proposed system consists of an improved structure from motion (SfM) approach. First, a novel multi-scale fast feature point detector (MFFPD) is proposed for detecting keypoints in the image in real-time with high accuracy. Secondly, a method is presented for sparse depth calculation at the detected key-points locations using multi-view 3D modeling. The proposed SfM system is capable of processing multiple video frames from a single planar or fisheye camera setup and is resilient to camera calibration parameter drifts. The algorithm pipeline is implemented using OpenCV/C++. Results are presented for sets of images that contain temporal motion and sets that contain lateral motion corresponding, respectively, to views from the front and side video cameras of a car.