Video annotation: Computers watching video

Abstract

Recently our research has focused on tools for video annotation — the generation of symbolic descriptions of dynamic scenes. In this paper I describe two specific examples of developing capabilities necessary for the understanding of action in video sequences. The first is a novel tracking technique that builds context-specific templates that are used only locally in time and space. The second example is some work on gesture recognition where time is implicitly represented in a probabilistic manner. We believe the future of computer vision lies in the processing of video and that while much work has been devoted to the representation and manipulation of static images, we are far behind in developing tools for considering action.