Temporal prediction of video sequences using an image warping technique based on color segmentation

Abstract

An image warping technique based on segmented regions is introduced for the temporal prediction of videophone-type sequences. At the encoder, a set of control points are determined from the previous frame and their corresponding best matched points are determined from the current frame. The selection process of these points is achieved by segmenting the previous frame into different regions using a color segmentation technique based on a recursive histogramming approach and the control points are subsequently chosen along the region boundaries. The spatial offset of these points between the previous and current frame are represented as motion vectors. The facial area is intraframe encoded to avoid distortions of the facial features which convey critical information. At the decoder, the same segmentation and control point selection algorithm is used along with the motion vectors in order to find the region boundaries of the predicted frame. The facial area is decoded and an affine transformation is finally used to determine the remaining regions and form the predicted frame. This technique produces results that are free from blocking artifacts as in the conventional block matching method, with only a moderate increase in computational complexity.