A Novel method for temporal action localization and recognition in untrimmed video based on time series segmentation

Abstract

Positioning of each action in a long complicated video is a challenging task in computer vision. To address this issue we propose a method with temporal boundary regression based on time series segmentation, which can generate proposals with flexible temporal duration. Firstly, we use a clustering algorithm to generate proposals, which is more efficient than sliding window method. It generates proposals by aggregating areas of high-probability behavior in time domain, and uses non-maximum suppression to remove redundancy. Then a multi-layer perceptron is used to refine boundary regression of behavior proposals, the process makes boundary coordinates closer to the real boundaries. Secondly, each behavioral proposal is represented by concatenating a three-subsegment feature description, which includes the proposal segment, its starting subsegment and its ending subsegment. Finally, the behavior proposal including a target action is identified by multi-layer perceptron. Our method is evaluated in two large data sets THUMOS14 and ActivityNet, which are commonly used in time series behavior detection task. The recognition rates can reach 30.1% and 33.19% respectively, which proves that the method can effectively improve the classification accuracy.