An Application of a Topic Model to Two Educational Assessments

Abstract

A topic model is a statistical model for extracting latent clusters or themes from the text in a collection of documents. The purpose of this study was to apply a topic model to two educational assessments. In the first study, the model was applied to students’ written responses to an extended response item on an English Language Arts (ELA) test. In the second study, a topic model was applied to the errors students’ made on a fractions computation test. The results for the first study showed five distinct writing patterns were detected in students’ writing on the ELA test. Two of the patterns were related to low scores, two patterns were associated with high scores and one pattern was unrelated to the score on the test. In the second study, five error patterns (i.e., latent topics) were detected on the pre-test and six error patterns were detected on the post-test for the fractions computation test. The results for Study 2 also yielded evidence of instructional effects on students’ fractions computation ability. Following instruction, more students in the experimental instruction condition made fewer errors than students in the business-as-usual condition.