Writing across the curriculum: Reliability testing of a standardized rubric

Abstract

Background: Rubrics positively affect student academic performance; however, accuracy and consistency of the rubric and its use is imperative. The researchers in this study developed a standardized rubric for use across an undergraduate nursing curriculum, then evaluated the interrater reliability and general usability of the tool. Method: Faculty raters graded papers using the standardized rubric, submitted their independent scoring for interrater reliability analyses, then participated in a focus group discussion regarding rubric use experience. Results: Quantitative analysis of the data showed a high interrater reliability (a =.998). Content analysis of transcription revealed several positive themes: Consistency, Emphasis on Writing Ability, and Ability to Use the Rubric as a Teaching Tool. Areas for improvement included use of value words and difficulty with point allocation. Conclusion: Investigators recommend effective faculty orientation for rubric use and future work in developing a rubric to assess reflective writing.