Deep Multi-task attribute-driven ranking for fine-grained sketch-based image retrieval

37Citations
Citations of this article
31Readers
Mendeley users who have this article in their library.

Abstract

Fine-grained sketch-based image retrieval (SBIR) aims to go beyond conventional SBIR to perform instance-level cross-domain retrieval: finding the specific photo that matches an input sketch. Existing methods focus on designing/learning good features for cross-domain matching and/or learning cross-domain matching functions. However, they neglect the semantic aspect of retrieval, i.e., what meaningful object properties does a user try encode in her/his sketch? We propose a fine-grained SBIR model that exploits semantic attributes and deep feature learning in a complementary way. Specifically, we perform multi-task deep learning with three objectives, including: retrieval by fine-grained ranking on a learned representation, attribute prediction, and attribute-level ranking. Simultaneously predicting semantic attributes and using such predictions in the ranking procedure help retrieval results to be more semantically relevant. Importantly, the introduction of semantic attribute learning in the model allows for the elimination of the otherwise prohibitive cost of human annotations required for training a fine-grained deep ranking model. Experimental results demonstrate that our method outperforms the state-of-the-art on challenging fine-grained SBIR benchmarks while requiring less annotation.

Cite

CITATION STYLE

APA

Song, J., Song, Y. Z., Xiang, T., Hospedales, T., & Ruan, X. (2016). Deep Multi-task attribute-driven ranking for fine-grained sketch-based image retrieval. In British Machine Vision Conference 2016, BMVC 2016 (Vol. 2016-September, pp. 132.1-132.11). British Machine Vision Conference, BMVC. https://doi.org/10.5244/C.30.132

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free