Boxless Action Recognition in Still Images via Recurrent Visual Attention

Abstract

Boxless action recognition in still images means recognizing human actions in the absence of ground-truth bounding boxes. Since no ground-truth bounding boxes are provided, boxless action recognition is more challenging than traditional action recognition tasks. Towards this end, AttSPP-net jointly integrates soft attention and spatial pyramid pooling into a convolutional neural network, and achieves comparable recognition accuracies even with some bounding box based approaches. However, the soft attention of AttSPP-net concentrates on only one fixation, rather than combining information from different fixations over time, which is the mechanism of human visual attention. In this paper, we take inspiration from this mechanism and propose a ReAttSPP-net for boxless action recognition. ReAttSPP-net utilizes a recurrent neural network model of visual attention in order to extract information from a sequence of fixations. Experiments on three public action recognition benchmark datasets including PASCAL VOC 2012, Willow and Sports demonstrate that ReAttSPP-net can achieve promising results and obtains higher recognition performance than AttSPP-net.