R-Trans: RNN Transformer Network for Chinese Machine Reading Comprehension

Abstract

Machine reading comprehension (MRC) has gained increasingly wide attention over the past few years. A variety of benchmark datasets have been released, which triggers the development of quite a few MRC approaches based on deep learning techniques. However, most existing models are designed to address English MRC. When applying them directly to Chinese documents, the performance often degrades considerably because of some special characteristics of Chinese, the inevitable segmentation errors in particular. In this paper, we present the RNN Transformer network to tackle the Chinese MRC task. To mitigate the influence of incorrect word segmentation and mine sequential information of whole sentences, deep contextualized word representations and bidirectional gated recurrent units networks are adopted in our model. The extensive experiments have been conducted on a very large scale Chinese MRC corpus, viz., the Les MMRC dataset. The results show that the proposed model outperforms the baseline and other prevalent MRC models notably, and established a new state-of-the-art record on the Les MMRC dataset.