The reliability of peer assessment in a final-year information systems course

Abstract

The perceived lack of reliability is one aspect of peer assessment that makes educators reluctant to use it in their teaching. This study was done among third‒year information systems students who had to assess two assignments of their peers by using rubrics. The rubrics differed in the sense that the one was more detailed requiring less subjective judgement while the other one was less detailed and required more subjective judgement from assessors. To determine the reliability of the peer assessment process, all the assignments of the participants in the study were re‒assessed by the lecturer and the marks compared. While the reliability of the peer assessment of both projects was acceptable, the assessment of the assignment requiring less judgement was more reliable compared to the assignment requiring more subjective judgement. A survey done among the participants revealed that they were happy with their marks and with the effort put in by their peers in assessing them. A significant number of students believed their peers awarded them higher marks for both assignments; however, this was only found to be the case for the assignment requiring more subjective judgment. It was also found that while the ratings of one peer rater resulted in reliable marks when the detailed rubric was used, it was not the case for the rubric requiring more subjective judgement. This study provides motivation for educators to use peer assessment as fears about the reliability thereof is unwarranted provided that properly designed rubrics are used, and students are adequately guided through the process. Keywords: Peer assessment, reliability, rubrics, accounting education