Video sequence querying using clustering of objects' appearance models

Abstract

In this paper, we present an approach for addressing the 'query by example' problem in video surveillance, where a user specifies an object of interest and would like the system to return some images (e.g. top five) of that object or its trajectory by searching a large network of overlapping or non-overlapping cameras. The approach proposed is based on defining an appearance model for every detected object or trajectory in the network of cameras. The model integrates relative position, color, and texture descriptors of each detected object. We present a 'pseudo track' search method for querying using a single appearance model. Moreover, the availability of tracking within every camera can further improve the accuracy of such association by incorporating information from several appearance models belonging to the object's trajectory. For this purpose, we present an automatic clustering technique allowing us to build a multi-valued appearance model from a collection of appearance models. The proposed approach does not require any geometric or colorimetric calibration of the cameras. Experiments from a mass transportation site demonstrate some promising results. © Springer-Verlag Berlin Heidelberg 2007.