Efficient goal attainment and engagement in a care manager system using unstructured notes

Abstract

Objective: To improve efficient goal attainment of patients by analyzing the unstructured text in care manager (CM) notes (CMNs). Our task is to determine whether the goal assigned by the CM can be achieved in a timely manner. Materials and Methods: Our data consists of CM structured and unstructured records from a private firm in Orlando, FL. The CM data is based on phone interactions between the CM and the patient. A portion of the data has been manually annotated to indicate engagement. We present 2 machine learning classifiers: An engagement model and a goal attainment model. Results: We can successfully distinguish automatically between engagement and lack of engagement. Subsequently, incorporating engagement and features from textual information from the unstructured notes significantly improves goal attainment classification. Discussion: Two key challenges in this task were the time-consuming annotation effort for engagement classification and the limited amount of data for the more difficult goal attainment class (specifically, for people who take a long time to achieve their goals). We successfully explore domain adaptation and transfer learning techniques to improve performance on the under-represented classes. We also explore the value of using features from unstructured notes to improve the model and interpretability. Conclusions: Unstructured CMNs can be used to improve accuracy of our classification models for predicting patient self-management goal attainment. This work can be used to help identify patients who may require special attention from CMs to improve engagement in self-management.