Identification of code-switched sentences and words using language modeling approaches

4Citations
Citations of this article
17Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

Globalization and multilingualism contribute to code-switching - the phenomenon in which speakers produce utterances containing words or expressions from a second language. Processing code-switched sentences is a significant challenge for multilingual intelligent systems. This study proposes a language modeling approach to the problem of code-switching language processing, dividing the problem into two subtasks: the detection of code-switched sentences and the identification of code-switched words in sentences. A code-switched sentence is detected on the basis of whether it contains words or phrases from another language. Once the code-switched sentences are identified, the positions of the code-switched words in the sentences are then identified. Experimental results show that the language modeling approach achieved an F -measure of 80.43% and an accuracy of 79.01% for detecting Mandarin-Taiwanese code-switched sentences. For the identification of code-switched words, the word-based and POS-based models, respectively, achieved F -measures of 41.09% and 53.08%. © 2013 Liang-Chih Yu et al.

Cite

CITATION STYLE

APA

Yu, L. C., He, W. C., Chien, W. N., & Tseng, Y. H. (2013). Identification of code-switched sentences and words using language modeling approaches. Mathematical Problems in Engineering, 2013. https://doi.org/10.1155/2013/898714

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free