Learning Image Representation Based on Convolutional Neural Networks

Abstract

Image similarity is widely applicable in image understanding and object tracking. It is easy for human to fulfill while difficult for machines. In this paper, we present a simple but efficient end-to-end mechanism to transfer an image into its corresponding representation in vector space based on Convolutional Neural Networks supervised by word2vec, which can then be applied to applications such as image classification and object detection, and a further work of image caption/description. We describe how we train the model to achieve a deep semantic understanding of the image along with its caption. We train our method on Flickr8k and Flickr30k datasets respectively, and evaluate on Corel1k benchmark dataset. Through the visualization of how our model extracts the features of images and produces similar vectors for similar images, we demonstrate the effectiveness of our proposed model.