Improving text-based image search with textual and visual features combination

Abstract

With the huge number of available images on the web, an effective image retrieval system has been more and more needed. Improving the performance is one of crucial tasks in modern text-based image retrieval systems such as Google Image Search, Frickr, etc. In this paper, we propose a unified framework to cluster and re-rank returned images with respect to an input query. However, owning to a difference to previous methods of using only either textual or visual features of an image, we combine the textual and visual features to improve search performance. The experimental results show that our proposed model can significantly improve the performance of a text-based image search system (i.e. Flickr). Moreover, the performance of the system with the combination of textual and visual features outperforms the performance of both the textual-based system and the visual-based system.