Semantic visual features in content-based video retrieval

Abstract

A new semantic visual features (e.g., car, mountain, and fire) navigation technology is proposed to improve the effectiveness of video retrieval. Traditional temporal neighbor browsing technology allows users to navigate temporal neighbors of a selected sample frame to find additional matches, while semantic visual feature browsing enables users to navigate keyframes that have similar features to the selected sample frame. A pilot evaluation was conducted to compare the effectiveness of three video retrieval designs that support 1) temporal neighbor browsing; 2) semantic visual feature browsing; and 3) fused browsing which is a combination of both temporal neighbor and semantic visual feature browsing. Two types of searching tasks: visual centric and non-visual centric tasks were applied. Initial results indicated that the semantic visual feature browsing system was more efficient for non-visual centric tasks.