Calibration Bias and the Interpretation of Clinical Learning Environment Perceptions Surveys

Abstract

Background: The clinical learning environment (CLE) is frequently assessed using perceptions surveys, such as the AAMC Graduation Questionnaire and ACGME Resident/Fellow Survey. However, these survey responses often capture subjective factors not directly related to the trainee's CLE experiences. Objective: The authors aimed to assess these subjective factors as "calibration bias" and show how it varies by health professions education discipline, and co-varies by program, patient-mix, and trainee factors. Methods: We measured calibration bias using 2011-2017 US Department of Veterans Affairs (VA) Learners' Perceptions Survey data to compare medical students and physician residents and fellows (n = 32 830) with nursing (n = 29 758) and allied and associated health (n = 27 092) trainees. Results: Compared to their physician counterparts, nursing trainees (OR 1.31, 95% CI 1.22-1.40) and allied/associated health trainees (1.18, 1.12-1.24) tended to overrate their CLE experiences. Across disciplines, respondents tended to overrate CLEs when reporting 1 higher level (of 5) of psychological safety (3.62, 3.52-3.73), 1 SD more time in the CLE (1.05, 1.04-1.07), female gender (1.13, 1.10-1.16), 1 of 7 lower academic level (0.95, 1.04-1.07), and having seen the lowest tercile of patients for their respective discipline who lacked social support (1.16, 1.12-1.21) and had low income (1.05, 1.01-1.09), co-occurring addictions (1.06, 1.02-1.10), and mental illness (1.06, 1.02-1.10). Conclusions: Accounting for calibration bias when using perception survey scores is important to better understand physician trainees and the complex clinical learning environments in which they train.