A Scene Classification Approach for Augmented Reality Devices

Abstract

Augmented Reality (AR) technology can overlay digital content over the physical world to enhance the user’s interaction with the real-world. The increasing number of devices for this purpose, such as Microsoft HoloLens, MagicLeap, Google Glass, allows to AR an immensity of applications. A critical task to make the AR devices more useful to users is the scene/environment understanding because this can avoid the device of mapping elements that were previously mapped and customized by the user. In this direction, we propose a scene classification approach for AR devices which has two components: i) an AR device that captures images, and ii) a remote server to perform scene classification. Four methods for scene classification, which utilize convolutional neural networks, support vector machine and transfer learning are proposed and evaluated. Experiments conducted using real data from an indoor office environment and Microsoft HoloLens AR device shows that the proposed AR scene classification approach can reach up to 99% of accuracy, even with similar texture information across scenes.