Key words extraction and semantic-based image retrieval on RNNs

Abstract

A quick glance at an image, it is very easy for us to perceive the main semantics while it is an challenge for computers. In this paper, we proposed an interesting approach to learn the core semantics of an image and generate the key words to describe it First, we need to get the corresponding of images and text named image-sentence alignment model, the alignment model was trained by image samples and the correlative sentence descriptions to learn the correspondences between the images and the texts. Second, we picked out several key words to describe the core semantic contents of images by the core semantic extraction method. Then, the semantic-based image retrieval was performed to demonstrate the effectiveness of the correspondence between the core semantic of image and the key word. The good performance of our method is illustrated on the Microsoft COCO dataset.