Time-delay recurrent neural network for cross-lingual speech recognition

Abstract

The speech recognition on low-resource language is a research hotspot. In this paper, on the basis of transfer learning, we propose a cross-lingual speech recognition system based on time-delay recurrent neural network. This network is comprised of deep belief time-delay neural network layer interleaved with long short-term memory recurrent neural network layer. We first train the whole merged network with a large labeled corpus of source language and afterward retrain the hidden layers using small target language corpus with both cross-entropy and lattice-free MMI objective function to enhance the recognition performance on the target language. Experiments show that the proposed system outperforms LSTM and DBN-DNN baseline system on THCHS30 corpus.