Spatio-Temporal Action Localization for Pedestrian Action Detection

Abstract

Current state-of-the-art temporal action detection methods are focused on untrimmed, multi-target videos. As popularized in the object detection framework, these methods perform classification on action class and detection of the duration for multiple instances. But these methods are unrealistic because the action of targets is usually irrelevant and complex in real-world. The previous methods utilize optical flow to handle multiple instances, but they cost too much time on estimating optical flow for evaluating. Inspired by spatio-temporal action detection, we improve the previous method with a new pedestrian action detection network which can detect a pedestrian in real-time. We replace Single Shot Multi-Box Detection (SSD) with RFB-Net which is more efficiency. The tube linking algorithm is introduced to link bounding boxes to different action instances. We use pedestrian action detection network to only process RGB frames which cost less time compared to two-stream based methods. Our framework achieves comparable result compared to the state-of-the-art and can detect in real-time.