Introducing a Swahili social media sentiment analysis dataset for the telecom industry

Abstract

Swahili is the most widely spoken language in Africa with over 200 million speakers. Despite its popularity in the continent, there is insufficient NLP research conducted on the language. The shortage of high-quality annotated datasets is attributed to this. In this paper, we introduce a Swahili dataset collected from Twitter, specifically designed to serve sentiment analysis tasks. The dataset comprised of a comprehensive collection of 8.7K tweets on products and services offered by telecommunications companies based in Tanzania. The tweets on the dataset are annotated manually by Swahili native speakers into three sentiments (Positive, Negative and Neutral). We have provided a detailed description of the steps involved in gathering and annotating the tweets, encompassing an elaborate account of the data collection method, annotation process, and dataset statistics. We tested the suitability of the developed dataset using five sentiment-classical machine learning models producing F1-scores ranging from 0.6889 to 0.7522 and 5 pre-trained transformer models producing F1-scores ranging from 0.7001 to 0.7306. Further, within this scholarly research paper, we expound upon the challenges encountered during the data collection and annotation processes. These challenges encompass bilingual tweets, the translation of emojis, the absence of Swahili language recognition by the Twitter platform, as well as the intricacies arising from Swahili words or phrases with multiple contextual meanings and informal vocabulary slang, and hashtag misclassification.