This paper presents a simple but effective approach to unknown word processing in Japanese morphological analysis, which handles 1) unknown words that are derived from words in a pre-defined lexicon and 2) unknown onomatopoeias. Our approach leverages derivation rules and onomatopoeia patterns, and correctly recognizes certain types of unknown words. Experiments revealed that our approach recognized about 4,500 unknown words in 100,000 Web sentences with only 80 harmful side effects and a 6% loss in speed.
CITATION STYLE
Sasano, R., Kurohashi, S., & Okumura, M. (2013). A Simple Approach to Unknown Word Processing in Japanese Morphological Analysis. In 6th International Joint Conference on Natural Language Processing, IJCNLP 2013 - Proceedings of the Main Conference (pp. 162–170). Asian Federation of Natural Language Processing. https://doi.org/10.5715/jnlp.21.1183
Mendeley helps you to discover research relevant for your work.