Inter-examiner reliability when using the Objective Structured Practical Examination (OSPE) mark sheet for physiotherapy practical

Abstract

The Objective Structured Practical Examination (OSPE) format is used during practical examinations as part of the physiotherapy undergraduate curriculum at the University of the Witwatersrand. Various factors influence inter-examiner reliability and investigating the inter-examiner reliability when using the OSPE can lead to improvement of the examination process. The aim of this study was to establish inter-examiner reliability when using the OSPE mark sheet. Methods: Twelve examiners participated in this study. Thirty-three second year PT students were examined at six stations and by two examiners at each station. The Spearman’s correlation test was used to establish inter-examiner reliability. Results: The general inter-examiner reliability of the OSPE mark sheet was high. There was a high correlation between examiners who had the same level of experience (r=0.79 to r=0.93; p<0.001). The background knowledge section of the OSPE mark sheet showed the greatest inter-examiner reliability (r=0.75 to r=0.91; p<0.001). Discussion: In general, a high inter-examiner reliability was found. Examiners with the same level of experience seemed to generally have better inter-examiner reliability when using the OSPE mark sheet. Furthermore, a well-described, operationalised list of micro-skills also improved inter-examiner reliability. Conclusion: The OSPE mark sheet aids inter-examiner reliability. The use of this method of examination should be encouraged.