Annotating Large Email Datasets for Named Entity Recognition with Mechanical Turk

  • Lawson N
  • Eustice K
  • 51


    Mendeley users who have this article in their library.
  • N/A


    Citations of this article.


Amazon's Mechanical Turk service has been successfully applied to many natural language processing tasks. However, the task of named entity recognition presents unique challenges. In a large annotation task involving over 20,000 emails, we demonstrate that a compet itive bonus system and interannotator agree ment can be used to improve the quality of named entity annotations from Mechanical Turk. We also build several statistical named entity recognition models trained with these annotations, which compare favorably to sim ilar models trained on expert annotations.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

There are no full text links


  • Nolan Lawson

  • Kevin Eustice

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free