Better foreground segmentation for 3D face reconstruction using graph cuts

Abstract

Research on image-based 3D reconstruction has recently shown a lot of good results, but it assumes precise target objects are already segmented from each input image. Traditionally, background subtraction was used to segment the target objects, but it can yield serious problems, such as noises and holes. To precisely segment the target objects, graph cuts have recently been used. Graph cuts showed good results in many engineering problems, as they can globally minimize energy functions composed of data terms and smooth terms, but it is difficult to automatically obtain prior information necessary for data terms. Depth information generated by stereo vision was used as prior information, which shows good results in their experiments, but it is difficult to calculate depth information for 3D face reconstruction, as the most of faces have homogeneous regions. In this paper, we propose better foreground segmentation method for 3D face reconstruction using graph cuts. The foreground objects are approximately segmented from each background image using background subtraction to assist to estimate data terms of energy functions, and noises and shadows are removed from the segmented objects to reduce errors of prior information. Removing the noises and shadows should cause to lose detail in the foreground silhouette, but smooth terms that assign high costs if neighboring pixels are not similar can fill out the lost silhouette. Consequently, the proposed method can segment more precise target objects by globally minimizing the energy function composed of smooth terms and approximately estimated data terms using graph cuts. © Springer-Verlag Berlin Heidelberg 2007.