Users' photos of items can reveal their tastes in a recommender system

Abstract

Recommender Systems (RS) are based on the generalization of the observed interactions of a population of users with a collection of items. Collaborative Filters (CF) give good results, but they degrade when there are few interactions to learn from. The alternative would be to observe some features of the users that could be linked to their tastes. However, specific information on users or items is often not available. In this research work, we explore how to exploit the photos of items taken by users. Our aim is to assign similar meanings to the photos of items with which the same group of users interacted. For this purpose, we define a multi-label classification task from images to sets of users. The classifier uses a general-purpose convolutional neural network to extract the basic visual features, followed by additional layers necessary to accomplish the learning task. To evaluate our proposal we compared it with CFs, using two tourism datasets that include: restaurants of six cities and points of interest of three locations. According to the experimentation carried out, the poor results achieved by CFs are outperformed by our proposal, which takes into account the visual and taste semantics of the available photos.