A Guide to Conducting Behavioral Research on Amazon ’ s Mechanical Turk

Abstract

Amazons Mechanical Turk is an online labor market where requesters post jobs and workers choose which jobs to do for pay. The central purpose of this paper is to demonstrate how to use this website for conducting behavioral research. We describe general techniques that apply to a variety of types of research and experiments across disciplines. We begin by discussing some of the advantages of doing experiments on Mechanical Turk, such as easy access to a large, diverse subject pool, low cost of doing experiments and faster iteration between theory development and executing experiments. We will discuss how the behavior of workers compares to experts and to laboratory subjects. Then, we illustrate the mechanics of putting a task on Mechanical Turk which involves recruiting subjects, executing the task, and reviewing the work that was submitted. We also provide solutions to common problems that a researcher might face in executing their research on this platform such as techniques for conducting synchronous experiments, methods to ensure high quality work, how to keep data private, and how to maintain code security.