Exploration of a possible relationship between examiner stringency and personality factors in clinical assessments: A pilot study

Abstract

Background: The reliability of clinical examinations is known to vary considerably. Inter-examiner variability is a key source of this variability. Some examiners consistently give lower scores to some candidates compared to other examiners and vice versa - the 'hawk-dove' effect. Stable examiner characteristics, such as personality factors, may influence examiner stringency. We investigated whether examiner stringency is related to personality factors. Methods: We recruited 12 examiners to view and score a video-recorded five station OSCE of six Year 1 undergraduate medical students at our institution. In addition examiners completed a validated personality questionnaire. Examiners' markings were tested for statistically significant differences using non-parametric one way analysis of variance. The relationship between examiners' markings and examiner personality factors was investigated using Spearman correlation coefficient. Results: At each station there was a statistically significant difference between examiners markings, confirming the presence of inter-examiner variability. Correlation analysis showed no association between stringency and any of the five major personality factors. When we omitted an outlier examiner we found a statistically significant negative correlation between examiner stringency and openness to experience with a correlation coefficients (rho) of - 0.66 (p = 0.03). Conversely there was a moderate positive correlation between examiner stringency and neuroticism with a correlation coefficient (rho) of 0.73 (p = 0.01). Conclusions: In this study we did not find any relationship between examiner stringency and examiner personality factors. However, following the elimination of an outlier examiner from the analysis, we found a significant relationship between examiner stringency and two of the big five personality factors (neuroticism and openness to experience). The significance of this outlier is not known. As this was a small pilot study we recommend further studies in this field to investigate if there is a relationship between examiner stringency in clinical assessments and personality factors.