Modelling the rate of trainees transitioning to Fellowship before achieving competence under the RANZCP’s Alternative Assessment Pathway to the Objective Structured Clinical Examination

Abstract

Objective: The Objective Structured Clinical Examination (OSCE) has been removed from the Royal Australian and New Zealand College of Psychiatrists’ (RANZCP) training pathway. This decision occurred in the context of an Alternative Assessment Pathway (AAP) necessitated by Covid-19, justified by logistical, methodological and equity concerns. The false positive rate of trainees progressing to Fellowship before achieving competence is a key indicator for evaluating any assessment leading to psychiatric Fellowship. Variations in the statistical properties of the AAP and OSCE were analysed for their impact on pre-competent trainees progressing to Fellowship. Method: Starting with the false positive scenario presented to justify discontinuing the OSCE, false positive rates associated with the AAP and OSCE were calculated based on different assumptions about reliability and accuracy. Results: The analyses suggest that less reliable and less accurate alternatives to the OSCE, such as the AAP, increase the number of pre-competent trainees progressing to Fellowship. Conclusions: Given possible increases in pre-competent trainees progressing to Fellowship while alternatives to the OSCE are finalised, confidence in the RANZCP's training program demands robust public analyses of those alternatives.