Image annotations based on semi-supervised clustering with semantic soft constraints

Abstract

An efficient image annotation and retrieval system is highly desired for the increase of amounts of image information. Clustering algorithms make it possible to represent images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words, have been published for image annotation. But most of these models cluster only using visual features, ignoring semantics of images. In this paper, we propose a novel model based on semi-supervised clustering with semantic soft constraints which can utilize both visual features and semantic meanings. Our method first measures the semantic distance with generic knowledge (e.g. WordNet) between regions of the training images with manual annotations. Then a semi-supervised clustering algorithm with semantic soft constraints is proposed to cluster regions with semantic soft constraints which are formed by semantic distance. The experiment results show that our model improves performance of image annotation and retrieval system. © Springer-Verlag Berlin Heidelberg 2006.