Proposed metrics for summarizing student evaluation of teaching data from balanced Likert scale surveys

Abstract

Student Evaluations of Teaching (SETs) gather crucial feedback on student experiences of teaching and learning and have been used for decades to evaluate the quality of teaching and student experience of instruction. In this paper, we make the case for an important improvement to the analysis of SET data that can further refine its interpretation. An extensive literature explores the reliability and consistency of SETs, and factors hypothesized to affect student evaluation of instructors. Much less attention has been given to the statistics used by institutions of higher education to summarize and report SET data—historically, arithmetic means and standard deviations computed from Likert scale scores. Here, we argue that a combination of two different statistics—interpolated median (IM) and percent favourable (PF) rating—together with a measure of dispersion suitable for ordinal data offer a fairer and more meaningful set of metrics for summarizing and interrogating SET data collected using balanced Likert scale surveys. To illustrate the power of these proposed metrics, we present an analysis of SET data from two academic years at a major Canadian university, and we also highlight an interesting and previously unreported relationship that we identified between the interpolated median and percent favourable rating.