Exploiting the limitations of spatio-temporal vision for more efficient VR rendering

Abstract

Increasingly higher virtual reality (VR) display resolutions and good-quality anti-aliasing make rendering in VR prohibitively expensive. The generation of these complex frames 90 times per second in a binocular setup demands substantial computational power. Wireless transmission of the frames from the GPU to the VR headset poses another challenge, requiring high-bandwidth dedicated links. Current approaches of rendering in VR with limited computational resources often utilise reprojection techniques [Beeler et al. 2016; Didyk et al. 2010; Vlachos 2016]. The assumption of these techniques is that the rendering cost can be reduced by drawing only every k-th frame, and generating in-between frames by transforming the previous frame. However, as reprojection often produces unexpected distortions and artifacts when encountering rapid brightness change, occlusions and repeated patterns, these techniques should only be used as a last resort. Vlachos [2016] suggests that displaying a lower resolution frame is often preferred. We propose a conceptually simple and robust technique for reducing both bandwidth and rendering cost for high-frame-rate displays by 25-49% with only marginal computational overhead and small impact on image quality. Our technique, Temporal Resolution Multiplexing (TRM), can also be applied to future high-refresh-rate desktop displays and television sets to improve motion quality.