Interactive human action search using body language

Abstract

Searching for human actions in a large video collection is a frequent demand in our daily lives. However, it is often not well supported by current multimedia technologies. For example, by using the traditional text-based search methods, it is not quite straightforward to give proper keywords as query input if users are uncertain about the textual or verbal descriptions of interesting actions in their mind. According to the sociological findings, the use of body language could be arguably a more natural and direct way for people to express their conscious or subconscious thoughts in a nonverbal manner. Therefore, in this paper, we propose an interactive system for human action search in videos, which is characterized by enabling the user to give a search query of interesting human actions through directly performing it. In contrast to a machines learning based recognition system, we address the problem of human action search with the approximate string matching (ASM) technique. As long as a user's actions can be matched with any sequence of the video database, they are said to be meaningful actions. The experiments demonstrate the effectiveness of our system in support of the user's search task. © 2012 IEEE.