AraBERT for Arabic cyberbullying detection in Facebook comments

Abstract

Cyberbullying is a significant issue on social media platforms. It poses serious emotional consequences and harassment to victims. Conventional pre-trained language models, such as Bidirectional Encoder Representations from Transformers (BERT), have achieved significant success in detecting cyberbullying through the analysis of natural language texts, especially with resource-rich languages such as English. However, for low-resource languages, such as Arabic, there has been limited attention given to the detection of cyberbullying. This research investigates the effectiveness of Arabic BERT (AraBERT), a pre-trained language model, for detecting Arabic cyberbullying comments. It also explores the trade-off between computational resources and model performance through various fine-tuning and freezing strategies. From an initial pool of >40 000 collected comments, we constructed a high-quality, balanced dataset of 20 000 Facebook comments written in Arabic. This subset was then manually labeled as either bullying or non-bullying to ensure data reliability and to facilitate robust model training. We employed fine-tuning techniques to adapt AraBERTv2 to the cyberbullying detection task. Through experimentation with layer freezing technique, we explored the trade-off between leveraging pre-trained knowledge and adapting the model to the specific task. Our findings demonstrate that fine-tuning all layers of AraBERTv2, which involves adjusting the weights and biases of each layer during training, achieved the highest performance. This approach offers a flexible method for applying a pre-trained model to new problems, resulting in an accuracy of 91.9% and an F1 score of 92.8%.