A Study on Moving Object Detection and Tracking with Partial Decoding in H.264|AVC Bitstream Domain

Abstract

An object detection and tracking technique has been an important issue traditionally in the field of computer vision and video processing since it enables efficient analysis of video contents. It can be utilized not merely for surveillance systems but also for interactive broadcasting services. However, most of current object detection and tracking techniques which utilize only raw pixel data are not practical due to tremendously high computa-tional complexity. Furthermore, most of videos tend to be communicated in the form of encoded bitstreams in order to enhance the transmission efficiency. In that case, the pixel domain approach requires additional computation time to ful-ly decode the encoded bitstream. In the meantime, H.264|AVC technology has been a popular compression tool for videos due to its high coding efficiency and the availability of its real-time encoding devices. Fortunately, the H.264|AVC bitstream contains encoded information such as motion vectors, residual data, and macroblock types which can be directly utilized as effective clues for object detection and tracking. The traditional compressed domain algorithms which make use of such encoded information have shown fast computation time with low computational complexity. However, these algorithms are available only under limited circumstances. In addition, they are difficult to be followed by the color extraction of objects or the object recognition which distinguishes one object from other objects. In this thesis, two methods for moving object detection and tracking with partial decoding in H.264|AVC bitstream domain are introduced. While one ap-proach is the semi-automatic method which users can initially select a target ob-ject in stationary or non-stationary scenes, another approach is the automatic method which all moving objects are automatically detected and tracked especially in stationary scenes. The former is beneficial to metadata authoring tools which generate additional contents like the position information of an object for the in-teractive broadcasting service. On the other hand, the latter is effective for sur-veillance systems with fixed cameras. Unlike conventional compressed domain algorithms, the proposed methods utilize partially decoded pixel data for object detection and tracking. Therefore, these methods show reliable performance in various scene situations as well as fast processing time enough to be performed in real-time. Also, these methods can support the color extraction of objects or the object recognition.