Visual quality assessment of stereoscopic image and video: Challenges, advances, and future trends

Abstract

Visual quality assessment of stereoscopic/3D images and videos has become an increasingly important and active field of research with the rapid growth in the quantity of stereoscopic/3D content created by the cinema, television, and entertainment industries. However, due to the diversity of stereoscopic/3D display technology and the complexity of human 3D perception, understanding the quality of experience (QoE) of stereoscopic/3D image and video is a difficult and multidisciplinary problem. Objective visual quality assessment attempts to quantify this subjective perception of visual QoE, utilizing tools from engineering, visual science, and psychology. In this chapter, first we discuss the challenges and difficulties one may face while trying to design and develop an effective objective quality assessment (QA) algorithm for stereoscopic images. This discussion is limited to “quality” where the stimulus being perceived is affected by some kind of distortions. In contrast to the success of a variety of objective QA algorithms for 2D images and videos, the field of stereoscopic image and video QA has been less successful in finding widely adopted quality measures. Most objective stereoscopic QA algorithms can be regarded as extensions of 2D QA algorithms, while few of them consider some aspects of depth perception and utilize either computed or measured depth/disparity information from the stereo pairs. We examine and analyze these stereoscopic QA algorithms, while focusing mainly on advances in exploiting natural scene statistics (NSS) and human visual system models in the design of stereoscopic QA algorithms. We also discuss recent work conducted on evaluating visual discomfort and fatigue when viewing stereoscopic images and videos—the more comprehensive “quality-of-experience” evaluation. Finally, we conclude the chapter with a discussion of possible future directions that the field of stereoscopic image and video QA may take. Our summary focuses on gaining a better understanding of depth/disparity sensation, using accurate and robust statistical models of natural stereo pairs, and performing a thorough analysis of various factors affecting the perception of stereoscopic distortions.