Continuous motion recognition using multiple time constant recurrent neural network with a deep network model

Abstract

Multiple timescale recurrent neural network (MTRNN) model is a useful tool to model a continuous signal for a dynamic task such as human action recognition. Different setting of initial states in the MTRNN brings us convenience to predict multiple signals using the same network model. On the contrary, optimal switching for suitable initial states in the slow context unit of the MTRNN becomes critical condition to achieve desired multiple dynamic tasks. In this paper, we propose a hybrid neural network model combining the MTRNN with a deep learning neural network (DN), which is to overcome the problem related to the initial state setting in the MTRNN. The DN together with MTRNN generates a suitable initial state for the slow context units in the MTRNN according to automatically detected situation change. We apply our approach to 20 motion skeleton units, which is obtained by KINECT, to construct three kinds of human motion sequences. The results show that the proposed method is able to recognize various motions using proper initial state information in a real-time procedure. © 2013 Springer-Verlag.