ERROR RESILIENT LAYERED STEREOSCOPIC VIDEO STREAMING
A. Serdar Tan1, Anil Aksay2, Cagdas Bilen2, Gozde Bozdagi Akar2 and Erdal Arikan1
1Electrical and Electronics Engineering Department, Bilkent University, Ankara, Turkey
2Electrical and Electronics Engineering Department, Middle East Technical University, Ankara, Turkey
ABSTRACT
In this paper, error resilient stereoscopic video streaming prob-
lem is addressed. Two different Forward Error Correction (FEC)
codes namely Systematic LT and RS codes are utilized to protect
the stereoscopic video data against transmission errors. Initially, the
stereoscopic video is categorized in 3 layers with different priorities.
Then, a packetization scheme is used to increase the efficiency of
error protection. A comparative analysis of RS and LT codes are
provided via simulations to observe the optimum packetization and
UEP strategies.
Index Terms— Forward error correction, video coding, stereo
vision
1. INTRODUCTION
Stereoscopic video transmission has gained considerable interest in
the past few years due to the increase in research and advances on 3-
D vision. Stereoscopic video is formed by the simultaneous capture
of two video sequences corresponding to left and right views of hu-
man visual system. The dependency of the left and right views can
be used to implement an efficient stereoscopic video codec. Once
coded, in order to transmit it over error prone channels, error robust
transmission methods are required.
Common error correction approaches for reliable transmission
of monoscopic video over packet networks utilize retransmissions
as in [1] or FEC methods as in [2], [3] and [4]. Retransmission
method brings large latency due to feedback messages that inform
the sender about the reliable reception of data. However, large la-
tency is unacceptable for video streaming applications. LT codes are
a novel retransmission-free and low-complexity FEC method intro-
duced in [5]. LT codes have gained attention in the video streaming
area in recent years [6].
Even though FEC codes are studied in depth for monoscopic
video, only a few studies exist for stereoscopic video [7]. In this
paper, we use RS and LT codes to protect the stereoscopic video
data against transmission errors. We define 3 layers for stereoscopic
video to be used for unequal error protection (UEP). We also present
a packetization scheme to increase the efficiency of error protection.
A comparative analysis of RS and LT codes are provided via simu-
lations to observe the optimum packetization and UEP strategies.
2. STEREOSCOPIC CODEC
In our experiments, multiview video codec based on H.264 [8] is
used. This codec uses a modified Decoded Picture Buffer (DPB) to
This work was supported by the EC under contract FP6-511568 3DTV
and in part by TUBITAK under contract BTT-Turkiye 105E065. A.S.T, A.A
and C.B are supported in part by TUBITAK (Scientific and Technical Re-
search Council of Turkey).

Stereo Encoder
Right Frame
Encoder
Source Left
Frame
Source
Right Frame
Left Frame
Encoder
Decoded
Picture
Buffer
Left
Frames
Right
Frames
Encoded
Left Frame
Encoded
Right Frame
Left Frame
Decoder
Right Frame
Decoder
Left
Frames
Decoded
Picture
Buffer
Decoded
Left Frame
Decoded
Right Frame
Right
Frames
Stereo Decoder
Encoded
Left Frame
Encoded
Right Frame
Fig. 1. Stereoscopic Encoder and Decoder Structure
perform both motion and disparity compensation with reduced com-
plexity. For stereoscopic videos, a special mode allows for mono-
scopic compatible streams, where standard H.264 decoders can de-
code only left frames and stereoscopic decoder can decode both left
& right frames. In monoscopic compatible mode, left frames are pre-
dicted from left frames only, whereas right frames can be predicted
from both left and right frames. Right frames are always predicted
from previous frames, whereas some of the left frames are encoded
without prediction (i.e. I-frames). Stereoscopic encoder and decoder
structure is given in Fig. 1.
Let I
L
,P
L
and P
R
denote the set of I-frames of left view, P-
frames of left views and P-frames of right views respectively. The set
of frames can be written in open form as I
L
= {I
L1
, I
L5
, ...}, P
L
=
{P
L2
, P
L3
, ...}, P
R
= {P
R1
, P
R2
, ...}, where i denotes the frame
number and L and R indicate the frames of left and right video. An
illustration is given in Fig. 2 where GOP size is set to 4.
Although this coding scheme is not layered, frames are not equal
in importance. We can classify the frames according to their contri-
bution to the overall quality and use them as layers of the video.
Since losing an I-frame causes large distortions due to motion / dis-
parity compensation and error propagation, I-frames should be pro-
tected the most. Among P-frames, left frames are more important