Egocentric scene reconstruction from an omnidirectional video

14Citations
Citations of this article
13Readers
Mendeley users who have this article in their library.

Abstract

Omnidirectional videos capture environmental scenes effectively, but they have rarely been used for geometry reconstruction. In this work, we propose an egocentric 3D reconstruction method that can acquire scene geometry with high accuracy from a short egocentric omnidirectional video. To this end, we first estimate per-frame depth using a spherical disparity network. We then fuse per-frame depth estimates into a novel spherical binoctree data structure that is specifically designed to tolerate spherical depth estimation errors. By subdividing the spherical space into binary tree and octree nodes that represent spherical frustums adaptively, the spherical binoctree effectively enables egocentric surface geometry reconstruction for environmental scenes while simultaneously assigning high-resolution nodes for closely observed surfaces. This allows to reconstruct an entire scene from a short video captured with a small camera trajectory. Experimental results validate the effectiveness and accuracy of our approach for reconstructing the 3D geometry of environmental scenes from short egocentric omnidirectional video inputs. We further demonstrate various applications using a conventional omnidirectional camera, including novel-view synthesis, object insertion, and relighting of scenes using reconstructed 3D models with texture.

References Powered by Scopus

MARCHING CUBES: A HIGH RESOLUTION 3D SURFACE CONSTRUCTION ALGORITHM.

8303Citations
N/AReaders
Get full text

Structure-from-Motion Revisited

4412Citations
N/AReaders
Get full text

Stereo processing by semiglobal matching and mutual information

3648Citations
N/AReaders
Get full text

Cited by Powered by Scopus

Progressively Optimized Local Radiance Fields for Robust View Synthesis

53Citations
N/AReaders
Get full text

VR-NeRF: High-Fidelity Virtualized Walkable Spaces

16Citations
N/AReaders
Get full text

Balanced Spherical Grid for Egocentric View Synthesis

12Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Jang, H., Meuleman, A., Kang, D., Kim, D., Richardt, C., & Kim, M. H. (2022). Egocentric scene reconstruction from an omnidirectional video. ACM Transactions on Graphics, 41(4). https://doi.org/10.1145/3528223.3530074

Readers' Seniority

Tooltip

Researcher 5

56%

PhD / Post grad / Masters / Doc 4

44%

Readers' Discipline

Tooltip

Computer Science 6

67%

Engineering 3

33%

Save time finding and organizing research with Mendeley

Sign up for free