Integrating a discrete motion model into GMM based background
subtraction
Christian Wolf Jean-Michel Jolion
Universite´ de Lyon, CNRS
INSA-Lyon, LIRIS, UMR5205, F-69621, France
fchristian.wolf,jean-michel.joliong@liris.cnrs.fr
Abstract
GMM based algorithms have become the de facto stan-
dard for background subtraction in video sequences,
mainly because of their ability to track multiple back-
ground distributions, which allows them to handle com-
plex scenes including moving trees, flags moving in the
wind etc. However, it is not always easy to deter-
mine which distributions of the mixture belong to the
background and which distributions belong to the fore-
ground, which disturbs the results of the labeling pro-
cess for each pixel. In this work we tackle this problem
by taking the labeling decision together for all pixels of
several consecutive frames minimizing a global energy
function taking into account spatial and temporal re-
lationships. A discrete approximative optical-flow like
motion model is integrated into the energy function and
solved with Ishikawa’s convex graph cuts algorithm.
1. Introduction
Background subtraction, the task of separating fore-
ground (object) pixels from background pixels in a
video, is an important step in many applications, ei-
ther because one is interested in an object’s silhouette
itself, or as a preprocessing step, for instance for track-
ing algorithms. Most existing methods build an explicit
background model either using a unimodal distribution
through median [2] or Kalman filtering [12] or similar
techniques, or a multi-modal distribution like GMM’s
[9, 11, 13]. A survey can be found in [8].
In some cases one is interested in a very precise
segmentation result, e.g. when an object’s shape shall
be used to recognize object classes or actions. In this
regard, the strengths of the existing methods are also
their weaknesses: the FG/BG segmentation decisions
are taken on a per pixel level, which is highly sub-
optimal.
In this paper we present a method 1 which improves
existing GMM based algorithms by taking the segmen-
tation decision on “global” level, i.e. simimultanuously
for all pixels of a whole block of the spatio-temporal
cube. Spatial and temporal interactions are taken into
account by a global energy function which is minimized
with graph cuts, searching for the exact globally best
solution. Temporal interactions handled with a motion
model which is calculated through an approximate op-
tical flow algorithm, also solved with graph cuts.
The contribution of this paper is twofold: the exper-
iments show a significant improvement over existing
methods. Furthermore, the algorithm for approximate
optical flow might be useful in other applications.
Our paper is organized as follows: section 2 is a short
reminder on GMM based BG subtraction. Section 3 de-
scribes our method and section 4 presents experimental
results. Section 5 finally concludes.
2. GMM based Background subtraction
Without loss of generality we present the case for
grayscale images in this paper, the adaptation to color
images and other multivariate cases is straightforward.
The perhaps most widely known and used back-
ground subtraction algorithm is the Stauffer-Grimson
algorithm [11] which is also be the base of our method.
It keeps K Gaussians for each pixel presenting a multi
modal distribution of pixel grayvalues. At each new
frame the new grayvalue y is checked against all Gaus-
sians and the best matching Gaussian is selected, if y is
within a threshold of standard deviations of the mean,
a new Gaussian is created else. The parameters of the
1This project was financed through the French National grant
“ANR-CaNaDA” — Comportements Anormaux: Analyse, De´tection,
Alerte, No. 128, which is part of the call for projects CSOSG 2006
Concepts Syste`mes et Outils pour la Se´curite´ Globale.