Dual-source backoff for enhancing language models

Abstract

This paper proposes a method of combining two n-gram language models to construct a single language model. One of the corpora is constructed from a very small corpus of the right domain of interest, and the other is constructed from a large but less adequate corpus. This method is based on the observation that a small corpus from the right domain has high quality n-grams but suffers from sparseness problem, while a large corpus from another domain is inadequately biased, but easy to obtain bigger size. The basic idea behind dual-source backoff is basically the same with Katz's backoff. We ran experiments with 3-gram language models constructed from newspaper corpora of several millions to tens of millions words together with models from smaller broadcast news corpora. The target domain was broadcast news. We obtained significant improvement by incorporating a small corpus around one thirtieth size of the newspaper corpus. © Springer-Verlag 2004.