Sample-based integrated background subtraction and shadow detection

Abstract

This paper presents an integrated background subtraction and shadow detection algorithm to identify background, shadow, and foreground regions in a video sequence, a fundamental task in video analytics. The background is modeled at pixel level with a collection of previously observed background pixel values. An input pixel is classified as background if it finds the required number of matches with the samples in the model. The number of matches required with the samples in the model to classify an incoming pixel as background is continuously adapted at pixel level according to the stability of pixel observations over time, thereby making better use of samples in dynamic as well as stable regions of the background. Pixels which are not classified as background in the background subtraction step are compared with a pixel-level shadow model. The shadow model is similar to the background model in that it consists of actually observed shadowed pixel values. Sample-based shadow modeling is a novel approach that solves the highly difficult problem of accurately modeling all types of shadows. Shadow detection by matching with the samples in the model exploits the recurrence of similar shadow values at pixel level. Evaluation tests on various public datasets demonstrate near state-of-the-art background subtraction and state-of-the-art shadow detection performance. Even though the proposed method contains shadow detection processing, the implementation cost is small compared with existing methods.