Comparative Study of Large Language Models as Emotion and Sentiment Analysis Systems: A Case-Specific Analysis of GPT vs. IBM Watson

Abstract

Sentiment analysis and emotion-detection techniques have wide applications in diverse fields. Various systems such as IBM Watson NLU have been developed for this purpose. Separately, large language models (LLMs) like GPT-3.5 have shown promise for diverse natural language processing (NLP) applications. This study investigates whether an LLM without explicit training could perform sentiment and emotion classification comparably to customized systems. For this purpose, a comparative analysis was conducted between GPT-3.5 and IBM Watson’s sentiment analysis, and emotion classification, using a dataset of 30,000 tweets related to the Covid-19 pandemic. The results revealed the versatility of LLMs, suggesting their potential transferability to diverse NLP tasks beyond their original training objective when properly prompted. GPT-3.5, despite not being explicitly trained for these tasks, achieves competitive performance with IBM Watson’s emotion classification capabilities when provided with a suitable prompting context. Precisely, GPT-3.5 demonstrates surprising adaptability to detect nuanced sentiments, such as irony, compared to Watson’s rigid emotion model. However, GPT-3.5 also struggles to fit textual expressions into the prescribed emotion classifications. Overall, this motivates expanded research into leveraging large pre-trained language models for affective computing applications by means of thoughtful prompt and evaluation design.