360RVW: Fusing Real 360° Videos and Interactive Virtual Worlds

1Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.
Get full text

Abstract

We propose a system to generate 360° realistic virtual worlds (360RVW) for the interactive spatial exploration of omnidirectional street-view videos. Our 360RVW enables users to explore photorealistic scenes with digital avatars, and interact with others. To create the virtual worlds our system only requires 360° videos with annotations of the start and end camera coordinate as input. We first detect street intersections to divide the input videos and remove the camera operator from the recordings using a video completion technique. Next, we analyze the 3D structure of the scene using semantic segmentation to define walkable areas. Finally, we render the environment using an ellipsoid projection surface to achieve a more realistic integration of the avatar into real-world 360° videos. The whole process is largely automated, enabling users to produce realistic and interactive virtual worlds without specialized skills or time-consuming manual interventions.

Cite

CITATION STYLE

APA

Takenawa, M., Sugimoto, N., Wöhler, L., Ikehata, S., & Aizawa, K. (2023). 360RVW: Fusing Real 360° Videos and Interactive Virtual Worlds. In MM 2023 - Proceedings of the 31st ACM International Conference on Multimedia (pp. 9379–9381). Association for Computing Machinery, Inc. https://doi.org/10.1145/3581783.3612670

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free