KGAT: An Enhanced Graph-Based Model for Text Classification

Abstract

As a fundamental task in natural language processing, text classification, which is to predict the class label of a given text, has been intensively studied. Consequently, a host of techniques have been developed, among which techniques that are based on graph neural network and its variant e. g., graph attention network (GAT ) achieved impressive performances, as they show superiority in dealing with complex graph-structured data. Despite effectiveness, most of these techniques suffer from several limitations, e. g., incapability in well-capturing correlation among words in a text. In light of these, we propose a comprehensive approach KGAT which incorporates multi-head GAT with enhanced attention and customized ReadOut operation for text classification. (1) Our approach constructs a text graph GT with edge weights from a text such that both semantic and structural information (with correlation degree) can be well captured. (2) On text graph GT, a novel attention mechanism is incorporated in a multi-head GAT for representation learning. (3) Our approach customizes ReadOut operation such that the representation of a text is refined by using a set of influential nodes of GT. Intensive experimental studies on both typical benchmark datasets and a newly created one (Sensitive ) show that our approach substantially outperforms other baseline methods and yields a promising technique for text classification.