Comparing Emotional Valence Scores of Twitter Posts from Manual Coding and Machine Learning Algorithms to Gain Insights to Refine Interventions for Family Caregivers of Persons with Dementia

Abstract

We randomly extracted Korean-language Tweets mentioning dementia/Alzheimer's disease (n= 12,413) from November 28 to December 9, 2020. We independently applied three machine learning algorithms (Afinn, Syuzhet, and Bing) using natural language processing (NLP) techniques and qualitative manual scoring to assign emotional valence scores to Tweets. We then compared the means and distributions of the four emotional valence scores. Visual examination of the graphs produced indicated that each method exhibited unique patterns. The aggregated mean emotional valence scores from the NLP methods were mostly neutral, vs. slightly negative for manual coding (Afinn 0.029, 95% CI [-0.019, 0.077]; Syuzhet 0.266, [0.236, 0.295]; Bing -0.271, [-0.289, -0.252]; manual coding -1.601, [-1.632, -1.569]). One-way analysis of variance (ANOVA) showed no statistically significant differences among the four means after normalization. These findings suggest that the application of NLP can be fairly effective in extracting emotional valence scores from Korean-language Twitter content to gain insights regarding family caregiving for a person with dementia.