Chinese Grammatical Error Correction Based on Convolutional Sequence to Sequence Model

Abstract

Chinese grammatical error correction (CGEC) is practically useful for learners of Chinese as a second language, but it is a rather challenging task due to the complex and flexible nature of Chinese language so that existing methods for English cannot be directly applied. In this paper, we introduce a convolutional sequence to sequence model into the CGEC task for the first time, since many Chinese grammatical errors are concentrated between three and four words and convolutional neural network can better capture the local context. A convolution-based model can obtain the representations of the context by fixed size kernel. By stacking convolution layers, long-term dependences can be obtained. We also propose two optimization methods, shared embedding and policy gradient, to optimize the convolutional sequence to sequence model through sharing parameters and reconstructing loss function. Besides, we collate the existing Chinese grammatical correction corpus in detail. The results show that the models we proposed two different optimization methods both achieve large improvement compared with the natural machine translation model based on a recurrent neural network.