A Sequence-to-Sequence Neural Network for Joint Aspect Term Extraction and Aspect Term Sentiment Classification Tasks

Abstract

Aspect-based Sentiment Analysis (ABSA) consists in extracting the terms or entities described in a text (attributes of a product or service) and the user perception of each aspect. Most earlier approaches are traditionally programmed sequentially, extracting the terms and then predicting their polarity. In this paper, we propose a joint sequence-to-sequence model that simultaneously extracts the terms and determines their polarities. The seq2seq architecture comprises an encoder, which can be an Arabic BERT model, and a decoder, which also can be an Arabic BERT, GPT, or BiGRU model. The encoder aims to preprocess the input sequence and encode it into a fixed-length vector called a context vector. The decoder reads that context vector from the encoder and generates the aspect term sentiment pair output sequence. We conducted experiments on two accessible Arabic datasets: Human Annotated Arabic Dataset (HAAD) of Book Reviews and The ABSA Arabic Hotels Reviews (ABSA Arabic Hotels). We achieve an accuracy score of 77% and 96% for HAAD and ABSA Arabic Hotels datasets respectively using BERT2BERT pairing. The results clearly highlight the superiority of the joint seq2seq model over pipeline approaches and the outperformance of BERT2BERT architecture over the pairing of BERT and BiGRU, and the pairing of BERT and GPT.