Thermal-Visible Video Fusion for Moving Target Tracking and Pedestrian Motion Analysis and Classification

Abstract

This chapter presents a novel system for pedestrian surveillance, including tasks such as detection, tracking, classification, and possibly activity analysis. The system we propose first builds a background model as a multimodal distribution of colors and temperatures. It then constructs a particle filter scheme that makes a number of informed reversible transformations to sample the model probability space to maximize posterior probability of the scene model. Observation likelihoods of moving objects account their three-dimensional locations with respect to the camera and occlusions by other tracked objects as well as static obstacles. After capturing the coordinates and dimensions of moving objects, we apply a classifier based on periodic gait analysis. To differentiate humans from other moving objects such as cars, we detect a symmetrical double-helical pattern in human gait. Such pattern can then be analyzed using the frieze group theory. The results of tracking in color and thermal sequences demonstrate that our algorithm is robust to illumination change and performs well in outdoor environments.