Attention-based two-layer long short-term memory model for automatic essay scoring

Abstract

We propose a neural network architecture-based automatic essay scoring model which contains a two-layer long short-term memory (LSTM) and an attention mechanism layer. The Google word vector dataset, which includes the richer word information and contextual information than the local-trained word vector dataset, is used to generate the embedding word vector of the input layer of the model by pre-training. The lower layer of the two-layer LSTM network captures the context semantic information and hidden context dependency, and the upper layer extracts the deeper context dependency. The attention mechanism layer focuses on the information extracted from the upper hidden layer of two-layer LSTM and calculates the attention probability to highlight the importance of key information in the text. The dataset used for automatic essay scoring task is provided by the Hewlett Foundation, and the quadratic weighted kappa coefficient is used as the evaluation index of the model. The experimental results show that the proposed method outperforms other automatic essay scoring baseline models such as bidirectional LSTM, SKIPFLOW-LSTM, and so on, in terms of the value of quadratic weighted kappa coefficient.