Recognizing text entailment via bidirectional LSTM model with inner-attention

Abstract

In this paper, we propose a sentence encoding-based model for recognizing text entailment (RTE). In our approach, the encoding process of sentences consists of two stages. Firstly, average pooling is used over word-level bidirectional LSTM (biLSTM) to generate a first-stage sentence representation. Secondly, attention mechanism is employed to replace average pooling on the same sentence for better representations. Instead of using target sentence to attend words in source sentence, we utilize the sentence’s first-stage representation to attend words appeared in itself, which is called “Inner-Attention” in our paper. Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus has proved the effectiveness of “Inner-Attention” mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin.