Deep Learning Based Semantic Video Indexing and Retrieval

Abstract

Vast amount of video stored in web archives makes their retrieval based on manual text annotations impractical. This study presents a video retrieval system capitalizing on image recognition techniques. The article discloses the details of implementation and empirical evaluation results for the system entirely based on features, extracted by convolutional neural networks. It is shown that these features can serve as universal signatures of the semantic content of the video and can be useful for implementing several types of multimedia retrieval queries defined in MPEG-7 standard. Further, the graph-based structure of the video index storage is proposed in order to efficiently implement complicated spatial and temporal search queries. Thus, technical approaches proposed in this work may help to build cost-efficient and user-friendly multimedia retrieval system.