Automatic student engagement in online learning environment based on neural turing machine

Abstract

With the continuous and rapid growth of online courses, online learners’ engagement recognition has become a novel research topic in the field of computer vision and pattern recognition. While a few attempts to automatic engagement recognition has been studied in the literature, learning a robust engagement measure is still a challenging task. To address it, we propose a new automatic engagement recognition method based on Neural Turing Machine in this paper. In particular, we firstly extract student’s eye gaze features, facial action unit features, head pose features, and body pose features respectively, then combine these multi modal features into the final feature of our recognition task. Moreover, we propose the engagement recognition framework based on the idea of Neural Turing Machine to learn the weight of each short video feature. In consequence, the feature fused by different weights will be applied to identify the students’ engagement in learning online courses. Empirically, we show improved performance over state of the art methods to automatic engagement recognition on DAiSEE dataset.