Factor analysis reveals student thinking using the mechanics reasoning inventory

Abstract

The Mechanics Reasoning Inventory (MRI) [1] is an assessment instrument specifically designed to assess strategic reasoning skills involving core concepts in introductory Newtonian mechanics. Being an assessment of higher order thinking (as opposed to declarative or rule-based procedural thinking), it is necessary to check whether or not the mental constructs underlying actual student responses correlate with the authors' domain classification, which is the subject of this paper. The instrument consists of three types of problems: Whether momentum or energy is conserved in a given situation and why, (partly inspired by the paired what/why questions in Lawson's Classroom Test of Scientific Reasoning), application of Newton's 2nd and 3rd law, and decomposing problems into parts (inspired by Van Domelen [2]'s Problem Decomposition Diagnostic). It has been administered 183 times in two MIT courses since 2009. Exploratory Factor Analysis (EFA) revealed that each Lawson pair of questions should be considered as one item, after which it identified four factors among the 21 questions that correspond reasonably well with the intended physics topics, and a fifth factor correlated with the concept of circular motion, a difficult topic for students (even though not viewed as a core principle by the designers). We discuss why 6 of the items classified under factors that differed from the expert assignments. There was no strong indication that the students answered each of different problem types similarly, which is a hallmark of students using novice heuristics rather than reasoning based on physical principles to answer the questions.