M


Tracker: A Multi-View Approach to Segmenting and
Tracking People in a Cluttered Scene
Anurag Mittal (anurag@cs.umd.edu) and Larry S. Davis
(lsd@cs.umd.edu)
Department of Computer Science
University of Maryland
College Park, MD 20742
Abstract.
When occlusion is minimal, a single camera is generally sufficient to detect and track
objects. However, when the density of objects is high, the resulting occlusion and lack of
visibility suggests the use of multiple cameras and collaboration between them so that an
object is detected using information available from all the cameras in the scene.
In this paper, we present a system that is capable of segmenting, detecting and tracking
multiple people in a cluttered scene using multiple synchronized surveillance cameras located
far from each other. The system is fully automatic, and takes decisions about object detection
and tracking using evidence collected from many pairs of cameras. Innovations that help us
tackle the problem include a region-based stereo algorithm capable of finding 3D points inside
an object knowing only the projections of the object (as a whole) in two views, a segmentation
algorithm using bayesian classification and the use of occlusion analysis to combine evidences
from different camera pairs.
The system has been tested using different densities of people in the scene. This helps
us determine the number of cameras required for a particular density of people. Experiments
have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.
Keywords: Multi-Camera Tracking, Region-Based Stereo, Grouping and Segmentation, Wide-
Baseline Stereo
1. Introduction
In this paper we address the problem of segmenting, detecting and tracking
multiple people using a multi-perspective video approach. In particular, we
are concerned with the situation when the scene being viewed is sufficiently
“crowded” that one cannot assume that any or all of the people in the scene
would be visually isolated from any vantage point. Figure 1 shows images
from a 6-perspective sequence that will be used to illustrate our algorithm.
Notice that in all four images, no single person is viewed in isolation- i.e. nei-
ther occludes another person nor is occluded by another person. We assume
that our cameras are calibrated, and that people are moving on a calibrated
ground plane. We also assume that the cameras are frame synchronized.
We present an algorithm that takes a unified approach to segmentation,
detection and tracking using multiple cameras. We neither detect nor track
objects from a single camera, or camera pair; rather evidence is gathered from
c

2002 Kluwer Academic Publishers. Printed in the Netherlands.
ijcv.tex; 4/10/2002; 22:10; p.1

Figure 1. Images from a 6-perspective sequence at a particular time instant.
multiple camera pairs and decisions of detection and tracking are taken at
the end by combining the evidence in a robust manner taking occlusion into
consideration. Also, we do not simply assume that a connected component
of foreground pixels corresponds to a single object. Rather, we employ a
segmentation algorithm to separate out regions belonging to different people.
This helps us handle the case of partial occlusion and allows us to track people
and objects in a cluttered scene where no single person is isolated in any view.
Good segmentation of people in a crowded scene is facilitated by models
of the people being viewed. Unfortunately, the problem of detecting and find-
ing the positions of the people requires accurate image segmentation in the
face of occlusions. Therefore, we take a unified approach to the problem and
2