Two person interaction recognition based on a dual-coded modified metacognitive (DCMMC) extreme learning machine

Abstract

Human action recognition has been an active research area for over three decades. However, state-of-the-art proposed algorithms are still far from developing error-free and fully-generalized systems to perform accurate interaction recognition. This work proposes a new method for two-person interaction recognition from videos, based on well-known cognitive theories. The main idea is to perform classification based on a theory of cognition known as dual coding theory. The theory states that human brain processes and represents two types of information to learn/classify data named analogue and symbolic codes, i.e. (verbal as analogue and visual as symbolic). To implement such a theory in a two-person interaction classification system, we exploit dense trajectories as analogue codes and a bag of words as symbolic codes which are two code types hypothesized in the theory. In addition to dual coding theory, we propose to implement a metacognitive classifier model which adds a metalevel with its own rules to perform more accurate training process. We also propose a modification in a metacognitive component to prevent cognitive interference well known as the Stroop effect. Evaluations on both datasets revealed that the method offers comparable recognition accuracy (95.6% for the SBU interaction dataset and 91.1% for the UT-interaction dataset).