Machine Learning for Identifying Emotional Expression in Text: Improving the Accuracy of Established Methods

Abstract

Expression of emotion has been linked to numerous critical and beneficial aspects of human functioning. Accurately capturing emotional expression in text grows in relevance as people continue to spend more time in an online environment. The Linguistic Inquiry and Word Count (LIWC) is a commonly used program for the identification of many constructs, including emotional expression. In an earlier study by Bantum and Owen (Psychol. Assess. 21:79–88, 2009), LIWC was demonstrated to have good sensitivity yet poor positive predictive value. The goal of the current study was to create an automated machine learning technique to mimic manual coding. The sample included online support groups, cancer discussion boards, and transcripts from an expressive writing study, which resulted in 39,367 sentence-level coding decisions. In examining the entire sample, the machine learning approach outperformed LIWC, in all categories outside of sensitivity for negative emotion (LIWC sensitivity = 0.85; machine learning sensitivity = 0.41), although LIWC does not take into consideration prosocial emotion, such as affection, interest, and validation. LIWC performed significantly better than the machine learning approach when removing the prosocial emotions (p =