Collecting fluency corrections for spoken learner english

Abstract

We present crowdsourced collection of error annotations for transcriptions of spoken learner English. Our emphasis in data collection is on fluency corrections, a more complete correction than has traditionally been aimed for in grammatical error correction research (GEC). Fluency corrections require improvements to the text, taking discourse and utterance level semantics into account: The result is a more naturalistic, holistic version of the original. We propose that this shifted emphasis be reflected in a new name for the task: 'holistic error correction' (HEC).We analyse crowdworker behaviour in HEC and conclude that the method is useful with certain amendments for future work.