Sketch based Image Retrieval using Learned KeyShapes (LKS)

Abstract

Sketch based image retrieval is a particular case of the image retrieval problem, in which a query is not a regular example image. Instead, the query is a hand-drawn sketch representing what the user is looking for. This kind of problem has a lot of applications, in particular when an example image is not available. For instance, in searching for design pieces in digital catalogs. The natural ambiguity of sketches as well as the poor skills of drawing make the problem very challenging, which is reflected in the low performance achieved by current methods. In this work, we present a novel method for describing sketches based on detecting mid-level patterns called learned keyshapes. Our experiments were performed in two datasets, one with 1326 images and the other with approximately 15k images. Our results show an increase of effectiveness around 17% on the smaller dataset and 98% on the larger one, which represent new state-of-the-art performance in the sketch based image retrieval domain. We also show that our method allows us to achieve good performance even when we use around 20% of the sketch content.