State-of-the-art and recent progress in hybrid HMM/ANN speech recognition

Abstract

It is by now very well known that for pattern classification problems Artificial Neural Networks (ANNs) can provide (good) estimates of posterior probabilities of output classes conditioned on the input. As a result of this property, a number of researchers over the last seven years have built systems, referred to as hybrid HMM/ANN systems, in which ANNs have been discriminativeiy trained to estimate emission probabilities for hidden Markov models (HMMs). Such systems have been proved, on controlled tests, to be both effective in terms of accuracy and efficient in terms of CPU and memory run-time requirements. After a short introduction to these HMM/ANN systems, a few of the extensions currently under investigation will be discussed. While initial HMM/ANN systems were discriminant only locally (at the HMM state level), the underlying theory will be briefly revisited to allow ANNs to be trained in a (unsupervised) globally discriminant manner (at the sentence level); this will also provide us with new insights into the general HMM/ANN approach. Secondly, a particular new speech recognition approach, referred to as multi-band or multi-stream speech recognition, which could directly benefit from hybrid HMM/ANN developments will be discussed. In this latter approach, the speech signal is decomposed into frequency sub-bands which are independently processed by different HMM/ANN systems (the ANN allowing to introduce more temporal information to compensate for the lack of frequency information) that are recombined in a linear or nonlinear (ANN) way at some temporal anchor points. Some of the very promising results achieved so far by this approach will be presented.