Action recognition using motion history information and convolutional neural networks

Abstract

Human action recognition is a key step in video analytics, with applications in various domains. In this work, a deep learning approach to action recognition using motion history information is presented. Temporal templates capable of capturing motion history information in a video as a single image, are used as input representation. In contrast to assigning brighter values to recent motion information, we use fuzzy membership functions to assign brightness (significance) values to the motion history information. New temporal templates highlighting motion in various temporal regions are proposed for better discrimination among actions. The features extracted by a convolutional neural network from the RGB and depth temporal templates are used by an extreme learning machine (ELM) classifier for action recognition. Evidences across classifiers using different temporal templates (i.e. membership function) are combined to optimize the performance. The effectiveness of this approach is demonstrated on MIVIA video action dataset.