An interactive semi-supervised approach for automatic image annotation

Abstract

Automatic image annotation (AIA) is an effective technique to bridge the semantic gap between low level image features and high level semantics. However, most of the existing AIA approaches failed to consider the use of unlabeled data. In this paper, we present an interactive semi-supervised approach for AIA by integrating graph propagation model and kernel canonical correlation analysis (KCCA) together. We aim to jointly utilize the keywords associated with labeled and selected unlabeled images to annotate the residual unlabeled images. Toward this goal, we firstly estimate the annotations of unlabeled images by the consistency-driven graph propagation model. Then, the KCCA is applied to seek the semantic consistency between the two concurrent visual and textual features. In addition, the unlabeled image with highest semantic consistency is selected into the training set. Thus, with the enlarged training set, the potential of the semantic consistency between visual and textual representations could be boosted. Some experiments carried out on two standard databases validate the effectiveness of the proposed method. © 2012 Springer-Verlag.