Vision based modeling for film and multimedia production

Abstract

Recovery of a three-dimensional (3-D) description of the world from images has been a central topic in computer vision from the early days of AI research and is currently becoming of great importance, particularly to the film and entertainment industries. In particular, this article addresses the problems of preproduction and postproduction for 3-D compositing of graphics in film and of building multimedia content for computer entertainment systems, both areas of burgeoning activity currently driven extensively with expensive skilled human labor. In this article, we discuss how the particular requirements for accuracy and generality of these applications impact the choices of vision techniques and how the interactive nature of production systems can be exploited to bypass the difficulties of segmentation and scene understanding that has prevented equally successful systems from arising in traditional AI contexts. We show actual results of both preproduction and postproduction processes in film and multimedia production, including preproduction model building, postproduction alignment of 3-D mattes into film, and a combination of preproduction and postproduction for inserting 3-D graphics into a video scene. The article concludes with a discussion of how the technology can be profitably merged with existing production practices and how it can be extended to provide further capabilities in production of entertainment media. © 1997 Taylor & Francis Group, LLC.