Research on Chinese word segmentation algorithm based on special identifiers

Abstract

Chinese information processing is a tedious and massive information processing engineering, Chinese word processing is that the whole project-based and one among the important aspects. This paper provides a word segmentation method based on special identifiers, and realizes a word segmentation system by combining the special identifier set with the modified two-character dictionary structure, before it carries out the comparison test for that system and other word segmentation systems by SOUGOU training corpus's test text. © 2011 Springer-Verlag.