Recurrent neural networks for hypotheses re-scoring

Abstract

We present our first results in applications of recurrent neural networks to Russian. The problem of re-scoring of equiprobable hypotheses has been solved. We train several recurrent neural networks on a lemmatized news corpus to mitigate the problem of data sparseness. We also make use of morphological information to make the predictions more accurate. Finally we train the Ranking SVM model and show that combination of recurrent neural networks and morphological information gives better results than 5-gram model with Knesser-Ney discounting.