Developing and validating a primary care EMR-based frailty definition using machine learning

Abstract

Introduction Individuals who have been identified as frail have an increased state of vulnerability, often leading to adverse health events, increased health spending, and potentially detrimental outcomes. Objective The objective of this work is to develop and validate a case definition for frailty that can be used in a primary care electronic medical record database. Methods This is a cross-sectional validation study using data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) in Southern Alberta. 52 CPCSSN sentinels assessed a random sample of their own patients using the Rockwood Clinical Frailty scale, resulting in a total of 875 patients to be used as reference standard. Patients must be over the age of 65 and have had a clinic visit within the last 24 months. The case definition for frailty was developed using machine learning methods using CPCSSN records for the 875 patients. Results Of the 875 patients, 155 (17.7%) were frail and 720 (84.2%) were not frail. Validation metrics of the case definition were: sensitivity and specificity of 0.28, 95% CI (0.21 to 0.36) and 0.94, 95% CI (0.93 to 0.96), respectively; PPV and NPV of 0.53, 95% CI (0.42 to 0.64) and 0.86, 95% CI (0.83 to 0.88), respectively. Conclusions The low sensitivity and specificity results could be because frailty as a construct remains underdeveloped and relatively poorly understood due to its complex nature. These results contribute to the literature by demonstrating that case definitions for frailty require expert consensus and potentially more sophisticated algorithms to be successful.