Examining the Reliability of a Culminating Teacher Education Assessment and Discovering Areas for Reform

Abstract

The purpose of this article is to provide an overview of the process used to examine the inter-rater reliability of the Teacher Work Sample (TWS) Scoring Rubric involved with the senior culminating experience for teacher candidates used at a large comprehensive university.     The study compared holistic and analytic scores reported by Student Teacher Seminar course instructors to those of trained participants to determine the consistency of ratings between the two groups.  The study resulted in several clear areas for revising the TWS for reliability and created a foundation for future revisions.  What may prove to be the most important finding of the study, however, is the need to examine the differences among scoring practices of raters because scoring varies among people.  Common errors include misinterpretation of scoring rubrics, prompts, the teaching and learning process, and even concepts such as revised Bloom’s Taxonomy.  This finding could be generalized to other universities as all education programs utilize scoring prompts and rubrics to measure teacher candidate performance and most all use  revised Bloom’s Taxonomy in the teaching and learning process.