Crowdsourcing Controls: A Review and Research Agenda for Crowdsourcing Controls Used for Macro-tasks

Abstract

Crowdsourcing---the employment of ad hoc online labor to perform various tasks---has become a popular outsourcing vehicle. Our current approach to crowdsourcing---focusing on micro-tasks---fails to leverage the potential of crowds to tackle more complex problems. To leverage crowds to tackle more complex macro-tasks requires a better comprehension of crowdsourcing controls. Crowdsourcing controls are mechanisms used to align crowd workers' actions with predefined standards to achieve a set of goals and objectives. Unfortunately, we know very little about the topic of crowdsourcing controls directed at accomplishing complex macro-tasks. To address issues associated with crowdsourcing controls for macro-tasks, this chapter has several objectives. First, it presents and discusses the literature on control theory. Second, this chapter presents a scoping literature review of crowdsourcing controls. Finally, the chapter identifies gaps and puts forth a research agenda to address these shortcomings. The research agenda focuses on understanding how to employ the controls needed to perform macro-tasking in crowds and the implications for crowdsourcing system designers.