A sub-picture-based omnidirectional video live streaming platform

Abstract

MPEG Omnidirectional Media Format (OMAF) specifies both a viewport-dependent video profile and a viewportdependent presentation profile to enable immersive media applications. A sub-picture-based approach for viewportdependent streaming is one of the main approaches being explored in MPEG OMAF standardization. This paper presents a sub-picture-based omnidirectional video live streaming platform with state of the art technologies integrated on both server and client sides to illustrate the benefits of such viewport-dependent omnidirectional video streaming approach. The technologies include omnidirectional video acquisition, sub-picture partitioning, real-time GPUaccelerated HEVC encoding and DASH-based live streaming. The presented platform supports virtual reality (VR) clients including both VR head-mounted displays (HMDs) and conventional 2D displays. Viewing orientation tracking and realtime viewport extraction are also supported in our platform. As with all live streaming platforms, one of the main goals of our platform is to minimize end-to-end system latency. A new metric called Comparable-Quality Viewport Switching (CQVS) latency is proposed to evaluate the performance of viewport dependent video streaming and presentation. The CQVS latency is defined as the amount of time it takes for the viewport video quality to improve to a level comparable to that prior to viewport switching. The platform was demonstrated in the Joint 3GPP and VRIF workshop on VR and the 2018 Mobile World Congress as one of the first OMAF-compliant viewport-dependent live streaming solutions.