Augmented reality shopping system through image search and virtual shop generation

Abstract

In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user’s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products’ life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants’ feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.