Polarity Classification of Low Resource Roman Urdu and Movie Reviews Sentiments Using Machine Learning-Based Ensemble Approaches

Abstract

The complex linguistic characteristics and limited resources present sentiment analysis in Roman Urdu as a unique challenge, necessitating the development of accurate NLP models. In this study, we investigate the performance of prominent ensemble methods on two diverse datasets of UCL and IMDB movie reviews with Roman Urdu and English dialects, respectively. We perform a comparative examination to assess the effectiveness of ensemble techniques including stacking, bagging, random subspace, and boosting, optimized through grid search. The ensemble techniques employ four base learners (Support Vector Machine, Random Forest, Logistic Regression, and Naive Bayes) for sentiment classification. The experiment analysis focuses on different N-gram feature sets (unigrams, bigrams, and trigrams), Chi-square feature selection, and text representation schemes (Bag of Words and TF-IDF). Our empirical findings underscore the superiority of stacking across both datasets, achieving high accuracies and F1-scores: 80.30% and 81.76% on the UCL dataset, and 90.92% and 91.12% on the IMDB datasets, respectively. The proposed approach has significant performance compared to baseline approaches on the relevant tasks and improves the accuracy up to 7% on the UCL dataset.