MuOE: A Multi-task Ordinality Aware Approach Towards Engagement Detection

Abstract

With the increasing adoption of online learning, decreasing student engagement is becoming rampant. Detecting this is the first step in making online education more viable and effective. We present MuOE, a Multi-task Ordinality-aware Engagement detection model to identify attention levels from students’ webcam videos. MuOE uses a transformer with exceptional sequence-processing capability and a novel selector-based attention mechanism that picks important video frames. Facial cue detection is used as an auxillary task in our multi-task formulation of the problem, so the shared model base has more supervision. We leverage the ordinal nature of engagement levels by introducing a smooth loss function that penalizes predictions based on closeness to the true label. In this paper, we motivate each component of MuOE, and demonstrate its utility through a set of quantative experiments. We achieve a state-of-the-art accuracy of 57.65% (Top-2 accuracy 95.07%) on the DAiSEE dataset.