Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk

  • Tetreault J
  • Filatova E
  • Chodorow M
  • 37

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract

In this paper we present results from two pilot studies which show that using the Amazon Mechanical Turk for preposition error annotation is as effective as using trained raters, but at a fraction of the time and cost. Based on these results, we propose a new evaluation method which makes it feasible to compare two error detection systems tested on different learner data sets.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

There are no full text links

Authors

  • Joel Tetreault

  • Elena Filatova

  • Martin Chodorow

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free