Improving Chinese word segmentation using partially annotated sentences

Abstract

Manually annotating is important for statistical NLP models but time-consuming and labor-intensive.We describe a learning task that can use partially annotated data as the training data. Traditional supervised learning task is a special case of such task. Particularly, we adapt the perceptron algorithm to train Chinese word segmentation models.We mix conventional fully segmented Chinese sentences with partially annotated sentences as the training data. Partially annotated sentences can be automatically generated from the heterogeneous segmented corpora as well as naturally annotated data such as markup language sentences like wikitexts without any additional manual annotating. The experiments show that our method improves the performances of both supervised model and semi-supervised models. © Springer-Verlag 2013.