Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale

Abstract

UNLABELLED: Background From 2010 to 2011, more than 70% of the clinical rotation competency evaluations for nephrology fellows in our program were rated "superior" using a 9-point Likert scale, suggesting some degree of "grade inflation." Objective We sought to assess the efficacy of a 5-point centered rotation evaluation in reducing grade inflation. Methods This retrospective cohort study of the impact of faculty education and a 5-point rotation evaluation on grade inflation was measured by superior item rating frequency and proportion of evaluations without superior ratings. The 5-point evaluation centered performance at the level expected for stage of training. Faculty education began in 2011-2012. The 5-point centered evaluation was introduced in 2012-2013 and used exclusively thereafter. A total of 68 evaluations, using the 9-point Likert scale, and 63 evaluations, using the 5-point centered scale, were performed after first-year fellow clinical rotations. Nine to 12 faculty members participated yearly. Results Faculty education alone was associated with fewer superior ratings from 2010-2011 to 2011-2012 (70.5% versus 48.3%, P = .001), declining further with 5-point centered scale introduction (2012-2013; 48.3% versus 35.6%; P = .012). Superior ratings declined with 5-point centered versus 9-point Likert scales (37.3% versus 59.3%, P = .001), specifically for medical knowledge, patient care, practice-based learning and improvement, and professionalism. On logistic regression, evaluations without superior scores were more likely for 5-point centered versus 9-point Likert scales (adjusted odds ratio [aOR] = 8.26; 95% CI 1.53-44.64; P = .014) and associated with faculty identifier (aOR= 1.18; 95% CI 1.03-1.35; P = .013), but not fellow identifier or training year quarter. CONCLUSIONS: Grade inflation was reduced with faculty education and the 5-point centered evaluation scale.