Regularized structured perceptron: A case study on chineseword segmentation, POS tagging and parsing

Abstract

Structured perceptron becomes popular for various NLP tasks such as tagging and parsing. Practical studies on NLP did not pay much attention to its regularization. In this paper, we study three simple but effective task-independent regularization methods: (1) one is to average weights of different trained models to reduce the bias caused by the specific order of the training examples; (2) one is to add penalty term to the loss function; (3) and one is to randomly corrupt the data flow during training which is called dropout in the neural network. Experiments are conducted on three NLP tasks, namely Chinese word segmentation, part-of-speech tagging and dependency parsing. Applying proper regularization methods or their combinations, the error reductions with respect to the averaged perceptron for some of these tasks can be up to 10%. © 2014 Association for Computational Linguistics.