Product recommendation through real-time object recognition on image classifiers

Abstract

With the development of e-commerce in the past years and its growing overlap over the classic way of doing business, many computational and statistical methods were researched and developed to make recommendations for products belonging to the store catalog. Often the data used in recommendation methods involves user interactions, being images and video types of information somewhat unexplored. This work, which we call Xanathar, proposes to extend such paradigm with real-time in-video recommendations for 25 classes of products, using image classifiers and feeding video streams to a modified ResNet-50 network processed on GPU, achieving a top-5 error of 5.17% and running at approximately 60 frames per second. Therefore, describing objects in the scene and proposing related products in-screen, directing user buying experience and creating an immersive and intensive purchase environment.