Mining students pre-instruction beliefs for improved learning

Abstract

In principle, learning can be increased by assessing the detailed state of student knowledge and mistaken knowledge with a pre-test and then optimizing instruction as measured by the post-test score. As a first step in this direction, we applied a Multidimensional Item Response Theory (MIRT) to 17,000 pre-instruction administrations of the Force Concept Inventory (FCI) to study students’ initial knowledge in detail. Examination of Item Response Curves (IRCs) showed that even students scoring below chance are not randomly guessing, but instead preferentially select only one or two distractors. Two dimensional IRT applied to the entire set of 150 possible responses, rather than applied dichotomously to the thirty questions, revealed two skill dimensions of comparable variance. Perpendicular directions were identified within this space corresponding to Newtonian ability and propensity to select responses whose IRC’s have a maximum at intermediate Newtonian ability rather than at the top of bottom of this dimension. These intermediate responses corresponded to known pre-Newtonian ideas, particularly the Medieval concept of impetus. The ability to measure the detailed misconceptions of individual students or classes will allow development and application of instructional interventions for such specific misunderstandings, which are typically unchanged by traditional instruction.