Analysis of Task Comparability in Digital Environment by the Case of Metacognitive Skills

Abstract

This article discusses the problem of task comparability with the help of scenario-based tasks for metacognitive skills. Using the data of “4С” tool for measuring critical thinking (N=500), the comparability of two scenarios within an identical digital environment with one set of indicators was investigated. The main difference in the scenarios lies in the contextual characteristics. The measurement invariance analysis of the instrument using confirmatory factor analysis was conducted. The results show that even with the equivalent construct structure and tasks' characteristics, the context of the scenario has an effect on the student`'s performance. The main differences in results were recorded for tasks involving interaction with the environment, where the test-taker created an object with elements. Tasks involving working with text in a digital environment can be considered comparable in case of elements content change. The possible reasons behind the observed differences in scenarios are discussed.