Identifying Metacognitive Processes Using Trace Data in an Open-Ended Problem-Solving Learning Environment

Abstract

Learning with open-ended learning environments (OELE) requires employing several metacognitive phases (e.g., planning and activation, monitoring) and processes (e.g., target goal setting, selection and adaptation of cognitive strategies). But, since such metacognitive phases and processes are challenging to execute by novice learners, it is essential to provide personalised feedback. Trace data is widely used to analyse metacognitive phases and processes, yet no existing research automatically identifies such phases/ processes using prediction models. This paper demonstrates the automatic prediction of four phases and five metacognitive processes using trace and think-aloud data of four learners interacting with a problem-solving OELE (MEttLE). We found that the Random Forest model trained using features extracted from learner interactions helps predict with a classification accuracy of up to 0.84, and Cohen’s kappa value that signifies fair to substantial agreement. We have used one-vs-all for multiclass classification with 10-fold cross-validation. Also, we applied SMOTE algorithm to upsample minority class instances to improve prediction models.