Convolutional neural networks (CNN) are widely used in many NLP tasks, which can employ convolutional filters to capture useful semantic features of texts. However, convolutional filters with small window size may lose global context information of texts, simply increasing window size will bring the problems of data sparsity and enormous parameters. To capture global context information, we propose to use the self-attention mechanism to obtain contextual word embeddings. We present two methods to combine word and contextual embeddings, then apply convolutional neural networks to capture semantic features. Experimental results on five commonly used datasets show the effectiveness of our proposed methods.
CITATION STYLE
Wu, X., Cai, Y., Li, Q., Xu, J., & Leung, H. fung. (2018). Combining Contextual Information by Self-attention Mechanism in Convolutional Neural Networks for Text Classification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 11233 LNCS, pp. 453–467). Springer Verlag. https://doi.org/10.1007/978-3-030-02922-7_31
Mendeley helps you to discover research relevant for your work.