Data sparseness problem is inherent and severe in language modeling. Smoothing techniques are usually widely used to solve this problem. However, traditional smoothing techniques are all based on statistical hypotheses without concerning about linguistic knowledge. This paper introduces semantic information into smoothing technique and proposes a similarity-based smoothing method which is based on both statistical hypothesis and linguistic hypothesis. An experiential iterative algorithm is presented to optimize system parameters. Experiment results prove that compared with traditional smoothing techniques, our method can greatly improve the performance of language model. © Springer-Verlag Berlin Heidelberg 2005.
CITATION STYLE
Xiao, J., Liu, B., Wang, X., & Li, B. (2005). A similarity-based approach to data sparseness problem of Chinese language modeling. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 3789 LNAI, pp. 761–769). https://doi.org/10.1007/11579427_77
Mendeley helps you to discover research relevant for your work.