Assessing Metacognitive Skills in Mathematical Problem Solving: A Comparative Study of a Task-Specific Questionnaire and Think Aloud Protocol

Abstract

Metacognitive skills play a crucial role in aiding students to comprehend when, why, where, and how to apply their knowledge to mathematical problem-solving. Consequently, teachers seek effective measurements to assess metacognitive skills and identify students with low metacognitive skills in mathematical problem-solving. Think Aloud Protocol (TAP) is recommended as the best method for measuring metacognitive skills in Mathematics. But using TAP is impractical for classes with a large number of students due to administrative challenges and time constraints. Questionnaires, although have less administrative issues, they are not generating reliable data comparable to TAP. To address these challenges, a task-specific questionnaire was designed based on the TAP, boasting a reliability of 0.89 (ordinal alpha) and a content validation score (S-CVI/Ave) of 0.9. The primary objective of this study is to examine whether the data generated by the questionnaire align through data generated by TAP. Twenty (20), third-year undergraduates from the Sri Lanka Technological Campus (SLTC) pursuing a BSc in Information Technology participated in the study. Engaging in the problem-solving process, they verbalized their thoughts, and immediately filled out the newly designed questionnaire. The overall correlation between the questionnaire and TAP (r=0.52) suggests the potential utility of this questionnaire as an alternative to TAP.