Validation of Electronic Health Record-Based Algorithms to Identify Specialist Palliative Care Within the Department of Veterans Affairs

Abstract

Background: The measurement of specialist palliative care (SPC) across Department of Veterans Affairs (VA) facilities relies on algorithms applied to administrative databases. However, the validity of these algorithms has not been systematically assessed. Measures: In a cohort of people with heart failure identified by ICD 9/10 codes, we validated the performance of algorithms to identify SPC consultation in administrative data and differentiate outpatient from inpatient encounters. Intervention: We derived separate samples of people by receipt of SPC using combinations of stop codes signifying specific clinics, current procedural terminology (CPT), a variable representing encounter location, and ICD-9/ICD-10 codes for SPC. We calculated sensitivity, specificity, and positive and negative predictive values (PPV, NPV) for each algorithm using chart review as the reference standard. Outcomes: Among 200 people who did and did not receive SPC (mean age = 73.9 years (standard deviation [SD] = 11.5), 98% male, 73% White), the validity of the stop code plus CPT algorithm to identify any SPC consultation was: Sensitivity = 0.89 (95% Confidence Interval [CI] 0.82–0.94), Specificity = 1.0 [0.96–1.0], PPV = 1.0 [0.96–1.0], NPV = 0.93 [0.86–0.97]. The addition of ICD codes increased sensitivity but decreased specificity. Among 200 people who received SPC (mean age = 74.2 years [SD = 11.8], 99% male, 71% White), algorithm performance in differentiating outpatient from inpatient encounters was: Sensitivity = 0.95 (0.88–0.99), Specificity = 0.81 (0.72–0.87), PPV = 0.38 (0.29–0.49), and NPV = 0.99 (0.95–1.0). Adding encounter location improved the sensitivity and specificity of this algorithm. Conclusions: VA algorithms are highly sensitive and specific in identifying SPC and in differentiating outpatient from inpatient encounters. These algorithms can be used with confidence to measure SPC in quality improvement and research across the VA.