Fine-tuning Transformer Models for Emotion Detection in Indonesian Tweets Indicating Depression

Abstract

Emotion detection on social media, especially related to mental health, has emerged as a significant topic in natural language processing (NLP). This study fine-tunes Transformer-based models ̶̶̶ IndoBERT, RoBERTa, and DistilBERT ̶̶̶ to identify emotions linked to depression in Indonesian tweets. A dataset of 5271 tweets was compiled using domain-specific keywords derived from mental health literature and expert inputs. Preprocessing follows standard NLP practices to ensure linguistic consistency and minimize noise. Several feature extraction techniques, including TF-IDF, Bag of Words (BoW), and Word2Vec, were evaluated using traditional machine learning (ML) classifiers under a 10-fold cross-validation scheme. The best performance was achieved by DistilBERT combined with TF-IDF and support vector machine (SVM), yielding an F1-score of 0.699 and accuracy of 0.705. Although the results remain moderate compared to high-resource languages, they mark a meaningful step toward modeling emotional indicators of depression in Indonesian social media contexts. The study underscores the importance of domain-specific and culturally aware approaches in advancing NLP applications for mental health.