Application of Nepali Large Language Models to Improve Sentiment Analysis

Abstract

With the rise in internet usage, Nepali individuals have left a flood of opinionated comments in their language on YouTube and other social media sites. Such remarks can be subjected to sentiment analysis, which can be useful for both research and business purposes. Such sentiment analysis models can be extremely useful in understanding the user's expectations towards the product which can uplift the business of any organization. Similarly, with the rise of Large Language models in the NLP space, there are several large language models pre-trained on the BERT architecture upon the Nepali text corpus. This research focuses on developing a benchmarking dataset for sentiment analysis in the Nepali language and demonstrating how large Nepali language models can be used to improve the results on downstream NLP tasks like sentiment analysis on such benchmark datasets. This paper describes an approach to how proper embeddings for a Nepali sentence can be extracted from the pre-trained Nepali language models. The comparison of transfer learning applied to the dataset on different machine learning and deep learning algorithms has been done in this study. From this experimentation, a state-of-the-art sentiment analysis model in the Nepali language with an F-score of 0.88 has been developed.