We propose a system to generate 360° realistic virtual worlds (360RVW) for the interactive spatial exploration of omnidirectional street-view videos. Our 360RVW enables users to explore photorealistic scenes with digital avatars, and interact with others. To create the virtual worlds our system only requires 360° videos with annotations of the start and end camera coordinate as input. We first detect street intersections to divide the input videos and remove the camera operator from the recordings using a video completion technique. Next, we analyze the 3D structure of the scene using semantic segmentation to define walkable areas. Finally, we render the environment using an ellipsoid projection surface to achieve a more realistic integration of the avatar into real-world 360° videos. The whole process is largely automated, enabling users to produce realistic and interactive virtual worlds without specialized skills or time-consuming manual interventions.
CITATION STYLE
Takenawa, M., Sugimoto, N., Wöhler, L., Ikehata, S., & Aizawa, K. (2023). 360RVW: Fusing Real 360° Videos and Interactive Virtual Worlds. In MM 2023 - Proceedings of the 31st ACM International Conference on Multimedia (pp. 9379–9381). Association for Computing Machinery, Inc. https://doi.org/10.1145/3581783.3612670
Mendeley helps you to discover research relevant for your work.