ElHBiAt: Electra Pre-Training Network Hybrid of BiLSTM, the Attention Layer to Aspect-Based Sentiment Analysis

Abstract

Aspect-based sentiment analysis is one of the famous and practical subjects in natural language processing. Traditional sentiment analysis assigns a polarity to the whole text or document and does not consider the aspects of the text, so it cannot correctly show the author’s opinion. Extracting aspects and determining their polarity has many challenges, such as the lack of rich datasets, negative transfer in cross-domain datasets, and fine-tuning of deep models of the ratio of texts with different contexts. In this research, a model has been proposed for aspect-based sentiment analysis with different contexts. This model uses the Electra as the backbone of the architecture, which is trained with different cross-domain datasets to transfer knowledge to the target domain. Also, in the proposed model of ElHBiAt, the resampling method is used to deal with imbalanced datasets, which causes the neural model to find less orientation than the polarities in the majority. In this model, two layers of attention are used to extract the essential aspects of the text, and the deep neural network BiLSMT is used to process the text in a two-way. The results show that the ElHBiAt model has been able to improve the efficiency by an average of 5.78% in the SemEval dataset, by 2% in the SEntFiN dataset, and by an average of 4.3% in the Drug dataset in terms of the F1-score.