The Effect of SMOTE Application on Support Vector Machine Performance in Sentiment Classification on Imbalanced Datasets

Abstract

This research explores the effect of applying Synthetic Minority Oversampling Technique (SMOTE) on the performance of Support Vector Machine (SVM) algorithm in sentiment classification on imbalanced datasets. Public review data was collected from social media platform X (formerly Twitter) regarding the Free Lunch Program, with a total of 2,368 reviews automatically labeled using the BERT model into three categories: positive, negative, and neutral. Sentiment imbalance in the dataset was addressed by applying SMOTE to generate synthetic data on minority classes. The research method follows the stages of Knowledge Discovery in Databases (KDD), including data selection, preprocessing, labeling, transformation using TF-IDF, SVM model training, and performance evaluation. The experimental results show that the application of SMOTE successfully improves the accuracy of the SVM model by 12.48%, from 71.41% to 83.89%. Other evaluation metrics, such as precision, recall, and F1-score, also showed significant improvement from 0.69, 0.71, and 0.68 to 0.84, respectively. These findings confirm that SMOTE is effective in overcoming data imbalance, resulting in a more accurate and reliable sentiment classification model. This research contributes to the application of sentiment analysis in data-driven public policy evaluation.