Dense 3D optical flow Co-occurrence matrices for human activity recognition

Abstract

In this paper, a new activity recognition technique is introduced based on the gray level co-occurrence matrices (GLCM) from a 3D dense optical flow of the input RGB and Depth videos. These matrices are one of the earliest techniques used for image texture analysis which are representing the distribution of the intensities and information about relative positions of neighboring pixels of an image. In this work, we propose a new method to extract feature vector values using the well-known Haralick features from GLCM matrices to describe the flow pattern by measuring meaningful properties such as energy, contrast, homogeneity, entropy, correlation and sum average to capture local spatial and temporal characteristics of the motion through the neighboring optical flow orientation and magnitude. To evaluate the proposed method and improve the activity recognition problem, we apply a recognition pipeline that involves the bag of local spatial and temporal features and three types of machine learning classifiers are used for comparing the recognition accuracy rate of our method. These classifiers are random forest, support vector machine and K-nearest neighbor. The experimental results carried on two well-known datasets (Gaming datasets (G3D) and Cornell Activity Datasets (CAD-60)), which demonstrate that our method outperforms the results achieved by several widely employed spatial and temporal feature descriptors methods.