Non-linear text regression with a deep convolutional neural network

Abstract

Text regression has traditionally been tackled using linear models. Here we present a non-linear method based on a deep convolutional neural network. We show that despite having millions of pa-rameters, this model can be trained on only a thousand documents, resulting in a 40% relative improvement over sparse lin-ear models, the previous state of the art. Further, this method is flexible allowing for easy incorporation of side information such as document meta-data. Finally we present a novel technique for interpreting the effect of different text inputs on this complex non-linear model.