Optimizing the Input of Physician Examiners in Setting Standards for a Large Scale OSCE: Experience with Part II of the Qualifying Examination of the Medical Council of Canada

Abstract

In 1994 and 1995, the Medical Council of Canada developed a new approach to setting the pass mark on its large scale, multi-center OSCE which is used to assess basic clinical and communication skills in physicians in Canada after 15 months of postgraduate medical training. The new approach utilized the global judgments of the physician examiners at each station to identify the candidates with borderline performances. The scores on those stations where candidates’ performances are judged to be borderline are summed for each station and a initial passing score is obtained for all stations and then the examination as a whole. The latter score is then adjusted upward one standard error of measurement for the final passing score. Based on the results presented, the new approach has worked very satisfactory to date. The advantages, disadvantages and areas of possible refinement for the approach are discussed.