CAPTAIN: Capsule Network Integrated With Multilingual Transformer for Textual Abusive Content Detection in Online Social Networks

Abstract

As online discussion and dialogue evolve, the proliferation of abusive content presents a substantial challenge to preserving a respectful and inclusive digital environment. Researchers are tackling the rise of abusive content on online social networks by developing more effective detection methods. Fine-tuning language models to identify derogatory and disturbing content has produced positive results. However, most efforts concentrate on high-resource languages, overlooking low-resource languages. This paper addresses this gap by proposing a method called CAPTAIN, which integrates a capsule network (CapsNet) with the multilingual Bidirectional encoder representations from transformers (mBERT). We select each component of CAPTAIN to enhance the understanding and classification of Urdu language text. CAPTAIN utilizes a publicly available multi-class dataset and outperforms both traditional machine learning and state-of-the-art methods on this low-resource Urdu dataset. This work also emphasizes the importance of balanced datasets for effectively detecting abusive content, especially in low-resource languages like Urdu, where class imbalance can significantly hinder model performance. The CAPTAIN achieves an F1-Score of 87% surpassing the state-of-the-art baselines by over 5% and it also achieves an average 96% ROC-AUC score for all classes. This study also examined how different hyperparameters affect performance, finding that more complex architectures and higher routing iteration rates decrease performance, while a less complex structure and higher capsule dimension rate improve classification performance.