Using topic modeling to understand comments in student evaluations of teaching

Abstract

Written comments in student evaluations of teaching offer a rich source of data for understanding instructors’ teaching and students’ learning experiences. However, most previous studies on student evaluations of teaching have focused on numeric ratings of close-ended questions, while few studies have tried to analyze the content of students’ written comments on open-ended questions, which normally involves a labor-intensive manual process of coding and categorizing. Such qualitative efforts prevent solutions on a large scale since it is almost impossible to go through all the textual data manually. Therefore, an innovative quantitative method that can analyze a large corpus of data holds great promise. This paper proposes the latent Dirichlet allocation (LDA) method of topic modeling to discover important themes that emerge in students’ written comments. We compare our results with findings in previous qualitative studies. We also investigate how these themes vary by course grade level and course subject. Our results provide evidence that topic modeling can be an effective and efficient alternative for understanding teaching and learning experiences through students’ written comments on a large scale.