This paper is a comparative study on representing units in Chinese text categorization. Several kinds of representing units, including byte 3-gram, Chinese character, Chinese word, and Chinese word with part of speech tag, were investigated. Empirical evidence shows that when the size of training data is large enough, representations of higher-level or with larger feature spaces result in better performance than those of lower level or with smaller feature spaces, whereas when the training data is limited the conclusion may be the reverse. In general, representations of higher-level or with larger feature spaces need more training data to reach the best performance. But, as to a specific representation, the size of training data and the categorization performance are not always positively correlated.
CITATION STYLE
Baoli, L., Yuzhong, C., Xiaojing, B., & Shiwen, Y. (2003). Experimental study on representing units in Chinese text categorization. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 2588, pp. 602–614). Springer Verlag. https://doi.org/10.1007/3-540-36456-0_67
Mendeley helps you to discover research relevant for your work.