Bi-Level Attention Model with Topic Information for Classification

Abstract

This paper proposes a bi-level attention neural network model (TBAM) that incorporates topic information. This model is suitable for a variety of classification situations and has proven effectiveness in sentiment classification and text classification. Compared with other advanced neural networks that are also used in classification, it has achieved better classification results. The TBAM model has three characteristics: (1) The model reflects the hierarchical structure of the document; (2) The model uses the attention mechanism at the vocabulary and sentence levels respectively, so that it can pay attention to the more important and the less important when constructing the document representation. content. (3) In the vocabulary level, by introducing potential topic information into the semantic representation of the vocabulary level to improve the effect of vocabulary representation, the topic information is regarded as domain information, which makes up for the ignorance of the domain information by the classification model, and increases the model domain adaptability. Through testing on the public data sets CCF-BDCI and THUCNews, the model shows good results on the three indicators of Precision, Recall, and F1.