A mobile, AR inside-out positional tracking algorithm, (MARIOPOT), suitable for modern, affordable cardboard-style VR HMDs

Abstract

Smartphone devices constitute a low-cost, mainstream and easy to use h/w for VR rendering and main component for modern, mobile VR Head- Mounted-Displays (HMDs). They support rotational tracking from on board sensors to manage orientation changes, via their Inertial Measurement Units (IMUs), but they lack positional tracking to reflect head translational movements, a key feature that modern, desktop VR HMDs nowadays provide out-of-the-box. Taking advantage of the RGB camera sensor that each modern mobile device is equipped, we describe a novel combination of inside-out AR tracking algorithms based on both marker and markerless tracking systems to provide the missing positional tracking for mobile HMDs. We employed this system as an affordable, low-cost VR visualization h/w and s/w method, for heritage professionals to employ it for VR archeological sites and Cultural Heritage related monuments interactive walk-throughs. We also compared our results with a recent holographic AR headset (Meta AR-glasses) that supports gesture recognition and interaction with the virtual objects via its RGB-D camera sensor and integrated IMU.