Reliability of the OSCE for physical and occupational therapists

Abstract

Purpose: To examine agreement rates between faculty members and clinical supervisors as OSCE examiners. Subjects: The study subjects were involved physical and occupational therapists working in clinical environments for 1 to 5 years after graduating from training schools as OSCE examinees, and a physical or occupational therapy faculty member and a clinical supervisor as examiners. Another clinical supervisor acted as a simulated patient. Methods: The agreement rate between the examiners for each OSCE item was calculated based on Cohen’s kappa coefficient to confirm inter-rater reliability. Results: The agreement rates for the behavioral aspects of the items were higher in the second than in the first examination. Similar increases were also observed in the agreement rates for the technical aspects until the initiation of each activity; however, the rates decreased during the middle to terminal stages of continuous movements. Conclusion: The results may reflect the recent implementation of measures for the integration of therapist education in training schools and clinical training facilities.