Motion compensated frame interpolation with a symmetric optical flow constraint

Abstract

We consider the problem of interpolating frames in an image sequence. For this purpose accurate motion estimation can be very helpful. We propose to move the motion estimation from the surrounding frames directly to the unknown frame by parametrizing the optical flow objective function such that the interpolation assumption is directly modeled. This reparametrization is a powerful trick that results in a number of appealing properties, in particular the motion estimation becomes more robust to noise and large displacements, and the computational workload is more than halved compared to usual bidirectional methods. The proposed reparametrization is generic and can be applied to almost every existing algorithm. In this paper we illustrate its advantages by considering the classic TV-L 1 optical flow algorithm as a prototype. We demonstrate that this widely used method can produce results that are competitive with current state-of-the-art methods. Finally we show that the scheme can be implemented on graphics hardware such that it becomes possible to double the frame rate of 640 x 480 video footage at 30 fps, i.e. to perform frame doubling in realtime. © 2012 Springer-Verlag.