Djangology: A Light-weight Web-based Tool for Distributed Collaborative Text Annotation

by Emilia Apostolova, Sean Neilan, Gary An, Noriko Tomuro, Steven Lytinen
Proceedings of the 7th Language Resources and Evaluation Conference (LREC 2010) ()


Manual text annotation is a resource-consuming endeavor necessary for NLP systems when they target new tasks or domains for which there are no existing annotated corpora. Distributing the annotation work across multiple contributors is a natural solution to reduce and manage the effort required. Although there are a few publicly available tools which support distributed collaborative text annotation, most of them have complex user interfaces and require a significant amount of involvement from the annotators/contributors as well as the project developers and administrators. We present a light-weight web application for highly distributed annotation projects - Djangology. The application takes advantage of the recent advances in web framework architecture that allow rapid development and deployment of web applications thus minimizing development time for customization. The application's web-based interface gives project administrators the ability to easily upload data, define project schemas, assign annotators, monitor progress, and review inter-annotator agreement statistics. The intuitive web-based user interface encourages annotator participation as contributors are not burdened by tool manuals, local installation, or configuration. The system has achieved a user response rate of 70% in two annotation projects involving more than 250 medical experts from various geographic locations.

Cite this document (BETA)

Readership Statistics

12 Readers on Mendeley
by Discipline
by Academic Status
25% Ph.D. Student
25% Student (Postgraduate)
8% Researcher (at an Academic Institution)
by Country
17% South Africa
8% India
8% France

Sign up today - FREE

Mendeley saves you time finding and organizing research. Learn more

  • All your research in one place
  • Add and import papers easily
  • Access it anywhere, anytime

Start using Mendeley in seconds!

Sign up & Download

Already have an account? Sign in