Developing peer review of instruction in an online master course model

Abstract

In this study we looked at how participation in a peer-review process for online Statistics courses utilizing a master course model at a major research university affects instructor innovation and instructor presence. We used online, anonymous surveys to collect data from instructors who participated in the peer-review process, and we used descriptive statistics and qualitative analysis to analyze the data. Our findings indicate that space for personal pedagogical agency and innovation is perceived as limited because of the master course model. However, responses indicate that participating in the process was overall appreciated for the sense of community it helped to build. Results of the study highlight the blurred line between formative and summative assessment when using peer review of instruction, and they also suggest that innovation and presence are difficult to assess through short term observation and through a modified version of a tool (i.e., the Quality Matters rubric) intended for the evaluation of an online course rather than the instruction of that course. The findings also suggest that we may be on the cusp of a second stage for peer review in an online master course model, whether in-person or online. Our findings also affirm the need for creating a sense of community online for the online teaching faculty. The experiences of our faculty suggest that peer review can serve as an integral part of fostering a departmental culture that leads to a host of intangible benefits including trust, reciprocity, belonging, and, indeed, respect.