Real Time Error Concealment in H.264 Video
Decoder for Embedded Devices
Amit Kale and Vallabha Hampiholi, Member IEEE.
MindTree Ltd.
Global Village RVCE Post, Mysore Road, Bangalore - 560 059
Email: {Amit Kale, Vallabha Hampiholi}@mindtree.com
Abstract—Transmission of compressed video over unreliable
network is an interesting topic of research. As compression is
inversely proportional to data redundancy. Highly compressed
data are hence more susceptible to errors. This paper discusses
error recovery and concealment techniques for H.264/AVC Base-
line Profile Decoder, where bandwidth, decoder performance and
memory are critical constraints for real time implementation.
This paper proposes a technique which gives improved perceptual
concealment without affecting the decoder performance and its
memory requirement. The technique is based on novel algorithm
which uses the available features in H.264 standard to perform
concealment, with minimal pre-processing. Comparison of the
novel algorithm has been made with the error concealment
capability of JM 13.2 reference software.
I. INTRODUCTION
H.264/Advanced Video Coding (AVC) [1] is the latest
standard in video compression. Compared to its predecessors
H.264/AVC provides better visual quality at lower bit-rates
making it ideal for network/wireless applications. Due to noisy
communication channels error concealment for video coded
data has become an important topic of research. The main
objective of Error Concealment is to hide the loss of video data
from the end user by concealing the corrupted/lost information
using previously decoded data. Packet loss, burst and bit errors
are common in noisy channels. Burst and bit errors can be fatal
as they cause H.264 video decoder to lose synchronization
and lead to its unpredictable behavior. Features like Slice-
structured coding, Flexible Macroblock Ordering (FMO), Ar-
bitrary Slice Order (ASO) and Data Partitioning make H.264
video stream more resilient to transmission errors [1], [2].
A key weakness of such methods, however, is it makes the
decoder dependent on the encoder to perform concealment
in case of errors. If these errors are not handled properly
it may lead to inconsistent behavior of the decoder and
may sometimes cause the decoder to crash under extreme
conditions. Thus it is imperative that the decoder detects all
transmission errors to perform good concealment.
In this paper a decoder-focused technique is proposed to
enhance error concealment through the use of neighboring
Macroblock (MB) motion information. This technique is well
suited for real time concealment of errors. In Section II of
this paper a review of existing techniques for concealing
errors in video transmission is presented. Section III discusses
the novel technique to perform error concealment (EC) on
erroneous H.264 video data. In Section IV performance of
the proposed algorithm with error concealment capability of
JM 13.2 reference software [3] is evaluated. While Section V
concludes the paper.
II. CURRENT TECHNIQUES
The simplest method to conceal lost MB is by copying
the co-located MB from the reference frame. This is done
by decoding the lost MB with motion vectors (MV) set to
zero. Although simple, this technique does not perform well
in video data streams with high motion content e.g., sports
video. Visual distortion can also be high, if the concealed
MBs are used as reference for decoding MBs in future frames.
In another approach [5] lost MBs are concealed using the
motion information estimated by averaging or by computing
median of the MVs for all the neighboring MBs. Experiments
show that this technique does not give better results [6], [7],
[8]. A Boundary Matching Algorithm (BMA) used in [6],
[7], [9] assumes high correlation among the neighboring
MBs which may not be true for videos with high motion
contents. Another error concealment algorithm based on BMA
is proposed in [10]. The best MV is found by minimizing a
cost function. This technique would increase the CPU cycles
to conceal the error, as repetitive decoding of a MB has to be
performed to obtain the distortion coefficient. Other proposed
techniques [11], [12], [13] are complex since they use max-
imum a posteriori (MAP) estimation, Markov Random fields
(MRF) and Lagrange interpolation to perform concealment.
These methods are thus not suitable to perform real-time
error concealment. Researchers in [14], [15] use MVs from
previous frames to estimate the lost MVs in current frame,
thereby increasing decoder memory requirement. The Pixel
based Overlapped Block Error Concealment (P-OBEC) and
Motion Vector OBEC (MV-OBEC) [16] provide satisfactory
concealment, but at the cost of increasing the overhead on the
decoder performance.
III. PROPOSED ALGORITHM
Most of the techniques discussed in previous section either
do not perform good concealment, or require offline processing
or need additional information. The algorithm proposed in this
paper does not require additional memory and performs satis-
factory concealment without degrading decoder performance.
In this method a neighbor search algorithm is applied to
check the availability of the macroblocks. Fig. 1 shows a