Recall Bias in Retrospective Assessment of Preoperative Patient Reported Outcomes

Abstract

Objectives: Patient reported outcomes (PROs) are measures of function, disability, and health status that may offer a unique assessment of provider quality and performance. The gold-standard method for collecting PROs is the prospective assessment of preoperative to postoperative change. This requires data collection before an intervention and then again after the intervention. This method is not always feasible due to unforeseen cases or emergencies, logistical and infrastructure barriers, and cost issues. In such cases a retrospective approach serves as a potential alternative. In this model, a patient is asked to complete an assessment about their perceived preoperative status during a time period sometime after the surgery. Although this method has its advantages, there is a particular risk of recall bias. There are conflicting conclusions regarding the reliability of the recalled preoperative PROs after orthopedic procedures. The aim of our study was to assess the agreement between prospectively and retrospectively collected PROs for a common, low-risk procedure. Methods: Patients that underwent arthroscopic rotator cuff repair between May 2012 and September 2017 at the study institution were identified. Inclusion criteria consisted of primary arthroscopic rotator cuff repair and preoperative prospectively collected American Shoulder and Elbow Surgeons Standard Shoulder Assessment Form (ASES) scores. All of the patients completed the ASES form preoperatively at their pre-assessment appointment. Patients were then contacted in the postoperative period and asked to recall their preoperative condition while completing another ASES form. Results: Seventy-one patients were included in this analysis (mean age 56.18 ± 10.48 years). The mean duration of symptoms from initial onset to the time of surgery was 8.54 ± 9.28 months. There was an average of 37.53 ± 17.02 months between the preoperative ASES and the recall ASES. Recall ASES scores were significantly lower than preoperative ASES scores (31.65 ± 16.87 vs 50.92 ± 19.57, p < 0.001). Less severe preoperative shoulder dysfunction was predictive of a greater difference between preoperative ASES and recall ASES (β = -0.60, R2 = 0.350, p < 0.001) (see Figure). Each 10-point increase in preoperative ASES score was predictive of a 6.04 point greater mismatch between preoperative and recall ASES. Likewise, a longer symptomatic period prior to surgery was associated with a greater ASES mismatch (R2 = 0.063, p = 0.029). The duration of time between surgery and recall was not a significant predictor of a difference between preoperative and recall ASES. Conclusion: This study demonstrated that there is poor agreement between prospectively and retrospectively collected preoperative PROs in the setting of rotator cuff surgery. Patient’s recalled ASES scores were significantly lower than their prospectively recorded ASES scores. This could lead to an overestimation of perceived benefit or effectiveness of the intervention. Our data supports prior studies that demonstrated that retrospective PROs are subject to recall bias and have been found to produce more favorable results than prospectively monitored health status data from the same patient. Our study supports the use of prospectively collected PROs and retrospective PROs should only be used in situations where baseline assessments are not possible.