Evaluating the Validity of Computerized Content Analysis Programs for Identification of Emotional Expression in Cancer Narratives

Abstract

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer (n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry and Word Count [LIWC] and Psychiatric Content Analysis and Diagnosis [PCAD]). Although the computerized coding methods captured most of the emotion identified by raters (LIWC sensitivity = .88; PCAD sensitivity = .83), both over-identified emotional expression (LIWC positive predictive value = .31; PCAD positive predictive value = .19). Correlational analyses suggested better convergent and discriminant validity for LIWC. The results highlight previously unrecognized deficiencies in commonly used computerized content-analysis programs and suggest potential modifications to both programs that could improve overall accuracy of automated identification of emotional expression. Although the authors recognize these limitations, they conclude that LIWC is superior to PCAD for rapid identification of emotional expression in text. © 2009 American Psychological Association.