Sentiment analysis on COVID tweets using COVID-Twitter-BERT with auxiliary sentence approach

Abstract

Sentiment analysis is a fascinating area as a natural language understanding benchmark to evaluate customers' feedback and needs. Moreover, sentiment analysis can be applied to understand the people's reactions to public events such as the presidential elections and disease pandemics. Recent works in sentiment analysis on COVID-19 present a domain-targeted Bidirectional Encoder Representations from Transformer (BERT) language model, COVID-Twitter BERT (CT-BERT). However, there is little improvement in text classification using a BERT-based language model directly. Therefore, an auxiliary approach using BERT was proposed. This method converts single-sentence classification into pair-sentence classification, which solves the performance issue of BERT in text classification tasks. In this paper, we combine a pre-trained BERT model from COVID-related tweets and the auxiliary-sentence method to achieve better classification performance on COVID tweets sentiment analysis. We show that converting single-sentence classification into pair-sentence classification extends the dataset and obtains higher accuracies and F1 scores. However, we expect a domain-specific language model would perform better than a general language model. In our results, we show that the performance of CT-BERT does not necessarily outperform BERT specifically in understanding sentiments.