Semantic video annotation by mining association patterns from visual and speech features

Abstract

In this paper, we propose a novel approach for semantic video annotation through integrating visual features and speech features. By employing statistics and association patterns, the relations between video shots and human concept can be discovered effectively to conceptualize videos. In other words, the utilization of high-level rules can effectively complement the insufficiency of statistics-based methods in dealing with broad and complex keyword identification in video annotation. Empirical evaluations on NIST TRECVID video datasets reveal that our proposed approach can enhance the annotation accuracy substantially. © 2008 Springer-Verlag Berlin Heidelberg.