Feature detection and tracking for visual effects: Augmented reality and video stabilization

Abstract

In this article, we propose two visual effects applications of feature tracking: augmented reality and video stabilization. The originality of the proposed methods lies in the combination and the adaptation of computer vision algorithms to visual effects applications in order to improve the process of visual effects in filmmaking. The first step involves feature detection and tracking. Different multi-scale feature detection algorithms have been tested: Harris-Laplace, Hessian-Laplace and Laplacian of Gaussian. An experiment was conducted to evaluate the tracking quantitatively. Then, we evaluate tracking for visual effects applications. For augmented reality, the result of tracking combined with perspective transformation are used to add an image to a planar surface of a scene filmed by a moving camera. We also evaluate the tracking for 3D objects insertion or matchmoving. For video stabilization, the result of tracking is used to estimate the geometrical transformation between frames.