Zero-Shot Fashion Products Clustering on Social Image Streams

Abstract

Computer Vision methods have been proposed to solve the problem of matching photographs containing some products from users in social media to products in retail catalogues. This is challenging due to the quality of the photographies, difficulties in dealing with garments and their category taxonomy. A N-Shot Learning approach is required as retail catalogues may contain hundreds of different products for which, in many cases, only one image is provided. This framework can be solved by means of Deep Metric Learning (DML) techniques, in which a metric to discriminate similar than dissimilar samples is learnt. The performance of different authors tackling this problem varies a lot but even if they perform reasonably well, the set of elements they need to return in order to include the exact product is large. As after the query there is a person curating the results, it is important to return the smallest set of elements possible, being ideally just to return only one: the related product. This paper proposes to solve the image-to-product image matching problem through a product retrieval system using DML and Zero-short Learning, focusing on garments, and applying some of the last advances on clustering techniques.