AI-based 360-degree video generation from monocular video for immersive experience

Abstract

We propose an artificial intelligence (AI)-based framework for generating 360-degree videos from videos recorded by monocular cameras. We also show immersive virtual reality content generation using AI through an analysis of user experience that compares manually designed and AI-generated 360-degree videos based on the proposed framework. The production of 360-degree videos conventionally requires special equipment, such as omni-directional cameras. Our framework is applicable to a massive amount of existing camera and video, hence it increases the availability of 360-degree videos. We implemented our framework in two steps. First, we generate a three-dimensional point cloud from the input video. Then, we apply AI-based methods to interpolate the sparse point cloud based on geometric and semantic information. Our framework will be applicable to several usages such as assisting surveying past traffic accident videos and education on showing the historical townscape of 360 degrees, etc.