Do you actually need an LLM? Rethinking language models for customer reviews analysis

Abstract

LLarge language models (LLMs) demonstrate strong natural language processing capabilities but come with significant computational costs, raising questions about their practical utility compared to small language models (SLMs). This study systematically compares SLMs (DistilBERT, ELECTRA) and LLMs (Flan-T5, Flan-UL2) on two customer review analysis tasks: sentiment polarity classification and product correlation analysis. Our results show that while LLMs outperform in sentiment classification, they do so at a much higher computational cost, whereas fine-tuned SLMs excel in domain-specific correlation analysis with greater efficiency. To balance accuracy and cost, we propose a context-enhanced hybrid (CE-Hybrid) model, which refines traditional hybrid methods by enriching LLM input with SLM-generated insights, reducing redundant computation while maintaining accuracy. Our findings quantify the trade-offs between model performance and resource efficiency, offering actionable insights for businesses to optimize AI deployment. These results have significant implications for real-world applications such as e-commerce, customer service automation, and business analytics.