Comparative Study of Large Language Models as Emotion and Sentiment Analysis Systems: A Case-Specific Analysis of GPT vs. IBM Watson

1Citations
Citations of this article
5Readers
Mendeley users who have this article in their library.
Get full text

Abstract

Sentiment analysis and emotion-detection techniques have wide applications in diverse fields. Various systems such as IBM Watson NLU have been developed for this purpose. Separately, large language models (LLMs) like GPT-3.5 have shown promise for diverse natural language processing (NLP) applications. This study investigates whether an LLM without explicit training could perform sentiment and emotion classification comparably to customized systems. For this purpose, a comparative analysis was conducted between GPT-3.5 and IBM Watson’s sentiment analysis, and emotion classification, using a dataset of 30,000 tweets related to the Covid-19 pandemic. The results revealed the versatility of LLMs, suggesting their potential transferability to diverse NLP tasks beyond their original training objective when properly prompted. GPT-3.5, despite not being explicitly trained for these tasks, achieves competitive performance with IBM Watson’s emotion classification capabilities when provided with a suitable prompting context. Precisely, GPT-3.5 demonstrates surprising adaptability to detect nuanced sentiments, such as irony, compared to Watson’s rigid emotion model. However, GPT-3.5 also struggles to fit textual expressions into the prescribed emotion classifications. Overall, this motivates expanded research into leveraging large pre-trained language models for affective computing applications by means of thoughtful prompt and evaluation design.

Cite

CITATION STYLE

APA

Carneros-Prado, D., Villa, L., Johnson, E., Dobrescu, C. C., Barragán, A., & García-Martínez, B. (2023). Comparative Study of Large Language Models as Emotion and Sentiment Analysis Systems: A Case-Specific Analysis of GPT vs. IBM Watson. In Lecture Notes in Networks and Systems (Vol. 842 LNNS, pp. 229–239). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-48642-5_22

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free