Examining the links between log data and reflective problem-solving practices in an interactive task

Abstract

Learning how to solve authentic problems is an important goal of education, yet how to assess and teach problem solving are research topics to be further explored. This study examines how interaction log data from a computerized task environment could be used to extract meaningful features in order to automate the assessment of reflective problem-solving practices. We collected survey responses and interaction log data of 40 college students working to solve the mass of a "mystery object"in an interactive physics simulation. The log data was parsed to reveal both the test trials conducted to solve the problem and the pauses in-between test trials, where potential monitoring and reflection of the problem-solving process took place. The results show that reflective problem-solving practices, as indicated by meaningful pauses, can predict problem-solving performance above and beyond participants' application of physics knowledge. Our approach to log data processing has implications for how we study problem solving using interactive simulations.