Gated convolutional networks for commonsense machine comprehension

Abstract

In this paper, we study the problem of commonsense machine comprehension and propose a new model based on convolutional neural networks and Gated Tanh-ReLU Units. The new model, which serves as an alternative to exiting recurrent models, consists of three layers: input layer, gated convolutional layer, and output layer. The input layer produces representations based on various features, such as part-of-speech and relation embeddings. Gated convolutional layer, the key component of our model, extracts n-gram features at different granularities and models the interactions between different texts (questions, answers, and passages). Bilinear interactions are used as output layer to capture the relations among the final expressive representations and to produce the final answers. We evaluate our model on the SemEval-2018 Machine Comprehension Using Commonsense Knowledge task. Experimental result shows that our model achieves highly competitive results with the state-of-the-art models but is much faster. To our knowledge, this is the first time a non-recurrent approach gains competitive performance with strong recurrent models for commonsense machine comprehension.