3D mesh construction from depth images with occlusion

Abstract

The realistic broadcasting is a broadcasting service system using multi-modal immersive media to provide clients with realism that includes such things as photorealistic and 3D display, 3D sound, multi-view interaction and haptic interactions. In such a system, a client is able to see stereoscopic views, to hear stereo sound, and even to touch both the real actor and virtual objects using haptic devices. This paper presents a 3D mesh modeling considering self-occlusion from 2.5D depth video to provide broadcasting applications with multi-modal interactions. Depth video of a real object is generally captured by using a depth video camera from a single point of view such that it often includes self-occluded images. This paper presents a series of techniques that can construct a smooth and compact mesh model of an actor that contains self-occluded regions. Although our methods work only for an actor with a simple posture, it can be successfully applied to a studio environment where the body movement of the actor is relatively limited. © Springer-Verlag Berlin Heidelberg 2006.