Identification of Postpartum Depression in Electronic Health Records: Validation in a Large Integrated Health Care System

Abstract

Background: The accuracy of electronic health records (EHRs) for identifying postpartum depression (PPD) is not well studied. Objective: This study aims to evaluate the accuracy of PPD reporting in EHRs and compare the quality of PPD data collected before and after the implementation of the International Classification of Diseases, Tenth Revision (ICD-10) coding in the health care system. Methods: Information on PPD was extracted from a random sample of 400 eligible Kaiser Permanente Southern California patients’ EHRs. Clinical diagnosis codes and pharmacy records were abstracted for two time periods: January 1, 2012, through December 31, 2014 (International Classification of Diseases, Ninth Revision [ICD-9] period), and January 1, 2017, through December 31, 2019 (ICD-10 period). Manual chart reviews of clinical records for PPD were considered the gold standard and were compared with corresponding electronically coded diagnosis and pharmacy records using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Kappa statistic was calculated to measure agreement. Results: Overall agreement between the identification of depression using combined diagnosis codes and pharmacy records with that of medical record review was strong (κ=0.85, sensitivity 98.3%, specificity 83.3%, PPV 93.7%, NPV 95.0%). Using only diagnosis codes resulted in much lower sensitivity (65.4%) and NPV (50.5%) but good specificity (88.6%) and PPV (93.5%). Separately, examining agreement between chart review and electronic coding among diagnosis codes and pharmacy records showed sensitivity, specificity, and NPV higher with prescription use records than with clinical diagnosis coding for PPD, 96.5% versus 72.0%, 96.5% versus 65.0%, and 96.5% versus 65.0%, respectively. There was no notable difference in agreement between ICD-9 (overall κ=0.86) and ICD-10 (overall κ=0.83) coding periods. Conclusions: PPD is not reliably captured in the clinical diagnosis coding of EHRs. The accuracy of PPD identification can be improved by supplementing clinical diagnosis with pharmacy use records. The completeness of PPD data remained unchanged after the implementation of the ICD-10 diagnosis coding.