Social network mining of requester communities in crowdsourcing markets

Abstract

Crowdsourcing is a new computing approach where human tasks are outsourced to a large number of human workers. Crowdsourcing has not only attracted attention from industry but also from various academic communities. Amazon Mechanical Turk (AMT) has been the first commercial platform offering crowdsourcing services to its customers. AMT is often referred to as a platform supplying ‘artificial’ artificial-intelligence. Recent research efforts have not been addressing the analysis of the community structure of large-scale crowdsourcing platforms. In this work, we discuss detailed statistics of the popular AMT marketplace to provide insights in task properties and requester behavior. Here we present a model to automatically infer requester communities based on task keywords. Hierarchical clustering is used to identify relations between keywords associated with tasks. We present novel techniques to rank communities and requesters by using a graph-based algorithm. Furthermore, we introduce models and methods for the discovery of relevant crowdsourcing brokers who are able to act as intermediaries between requesters and platforms such as AMT.