The Complexity of Crowdsourcing: Theoretical Problems in Human Computation

  • Kulkarni A
  • 75


    Mendeley users who have this article in their library.
  • N/A


    Citations of this article.


What does theoretical computer science have to say about human computation? We identify three problems at the intersection of crowdsourcing, operations research, and theoretical computer science whose solution would have a major impact on the design, evaluation, and construction of real crowdsourcing systems. In some cases, these problems can let us sidestep apparently difficult HCI challenges by making appropriate choices at the algorithmic level. In other contexts, theoretical tools provide a formal basis for evaluating the performance of algorithms and classifying the difficulty of tasks in crowdsourcing. Our problems are illustrated through two recent projects. The first, Turkomatic, is an attempt to construct a “universal” algorithm for generating workflows on microtask crowdsourcing platforms. The second, MobileWorks, is a new crowdsourcing engine designed from the bottom up to provide appropriate abstractions between the theoretical elements of human computation systems and interface/design questions. It is hoped that this analysis can spur the development of theoretical frameworks for understanding algorithms involving human computation.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document


  • Anand Kulkarni

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free