Image Captioning and Comparison of Different Encoders

Abstract

Generation of a sentence given an image, called image captioning, has been one of the most intriguing topics in computer vision. It incorporates knowledge of both image processing and natural language processing. Most of the current approaches integrates the concepts of neural network. Different predefined convolutional neural network (CNN) models are used for extracting features from an image and uni-directional or bi-directional recurrent neural network (RNN) for language modelling. This paper discusses about the commonly used models that are used as image encoder, such as Inception-V3, VGG19, VGG16 and InceptionResNetV2 while using the uni-directional LSTMs for the text generation. Further, the comparative analysis of the result has been obtained using the Bilingual Evaluation Understudy (BLEU) score on the Flickr8k dataset.