Towards retrieval of visual information based on the semantic models

Abstract

Most users want to find visual information based on the semantics of visual contents such as a name of person and an action happening in a scene. However, techniques for content-based image or video retrieval are not mature enough to recognize visual semantic completely. This paper concerns the problem of automated visual content classification that allows semantic exploration of the visual information. To enable semantic based image or visual object retrieval, we propose a new image representation scheme called visual context descriptor (VCD) that is a multidimensional vector in which each element represents the frequency of a unique visual property of an image or a region. VCD utilizes the predetermined quality dimensions (i.e., types of features and quantization levels) and semantic model templates mined in priori. Techniques for creating symbolic representation (called visual term) of visual content and semantic model profile mining and matching have also been explored. The proposed model classification technique utilizes contextual relevance of a visual term to a target semantic class in visual object discrimination. Contextual relevance of a visual cue to a semantic class is determined by using correlation analysis of ground truth samples.