Generalized relevance models for automatic image annotation

Abstract

This paper presents a generalized relevance model for automatic image annotation through learning the correlations between images and annotation keywords. Unlike previous relevance models, the proposed model can perform keyword propagation not only from the training images to the test ones but also among the test images. We further give a convergence analysis of the iterative algorithm inspired by the proposed model. Moreover, our spatial Markov kernel is used to define the inter-image relations for the estimation of the joint probability of observing an image with possible annotation keywords. This kernel was originally designed for image classification, and here we apply it to image annotation. The main advantage of using our spatial Markov kernel is that we can capture the intra-image context based on 2D Markov models, which is different from the traditional bag-of-words methods. Experiments on two standard image databases demonstrate that the proposed model outperforms the state-of-the-art annotation models. © 2009 Springer-Verlag Berlin Heidelberg.