Incorporating context-relevant knowledge into convolutional neural networks for short text classification*

Abstract

Some text classification methods don't work well on short texts due to the data sparsity. What's more, they don't fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the context-relevant concept features. The experimental results on three datasets show that our proposed model outperforms the state-of-the-art models.