A Self-attention Based LSTM Network for Text Classification

Abstract

Neural networks have been used to achieve impressive performance in Natural Language Processing (NLP). Among all algorithms, RNN is a widely used architecture for text classification tasks. The main challenge in sentiment classification is the quantification of the connections between context words in a sentence. Even though various types and structures of model have been proposed, they encounter the problem of gradient vanishing and are unlikely to show the full potential of the network. In this work, we present a new RNN model based on the self-attention mechanism to improve the performance while dealing with long sentences and whole documents. Empirical results show that our model outperforms the state-of-art algorithms.