Effectiveness of video ontology in query by example approach

Abstract

In this paper, we develop a video retrieval method based on Query-By-Example (QBE) approach where a query is represented by providing example shots. Relevant shots to the query are then retrieved by constructing a retrieval model from example shots. However, one drawback of QBE is that a user can only provide a small number of example shots, while each shot is generally represented by a high-dimensional feature. In such a case, a retrieval model tends to be overfit to feature dimensions which are specific to example shots, but are ineffective for retrieving relevant shots. As a result, many clearly irrelevant shots are retrieved. To overcome this, we construct a video ontology as a knowledge base for QBE-based video retrieval. Specifically, our video ontology is used to select concepts related to a query. Then, irrelevant shots are filtered by referring to recognition results of objects corresponding to selected concepts. Lastly, QBE-based video retrieval is performed on the remaining shots to obtain a final retrieval result. The effectiveness of our video ontology is tested on TRECVID 2009 video data. © 2011 Springer-Verlag.