Thread-safe: Towards recognizing human actions across shot boundaries

Abstract

We study the task of recognizing human actions in video whilst paying attention to the shot and thread editing structure. Most existing action recognition algorithms ignore this structure, but it is generally present in edited TV and film material. To this end, we make the following contributions: first, we introduce a new dataset of human actions to study the occurrence/reoccurrence of patterns of human actions in edited TV material; second, we propose composing a video into threads of related shots, removing some of the discontinuities due to shot boundaries; and third, we show the benefits of utilizing video threads in recognizing human actions. The experiments demonstrate that human action retrieval accuracy can be improved using threads.