High-level video semantic concept detection based on multi-level feature representations

Abstract

Semantic concept detection is a fundamental problem with many practical applications such as concept-based video retrieval. The major challenge of concept detection lies in the existence of the well-known semantic gap between the low-level visual features and the user's semantic interpretation of visual data. To bridge the semantic gap, we propose to promote the low-level visual feature to the middle-level representation, expecting that the underlying semantic aspects of image data can be discovered, and such latent aspects can better model the semantic of images. Specifically, latent Dirichlet allocation (LDA) is adopted to cluster the image data into semantic topics and the distributions on such topics are used as the middle-level feature vectors of the image. Meanwhile, a recently developed more efficient probabilistic representation of low-level features, i.e., Fisher Vector (FV) is used to complement the LDA representation for video concept detection. The experimental results on the TRECVID 2013 Semantic Indexing dataset have demonstrated the effectiveness of the proposed approach. © Springer International Publishing Switzerland 2013.