CrowdUtility: A Recommendation System for Crowdsourcing Platforms

Abstract

Crowd workers exhibit varying work patterns, expertise, and quality leading to wide variability in the performance of crowdsourcing platforms. The onus of choosing a suitable platform to post tasks is mostly with the requester, often leading to poor guarantees and unmet requirements due to the dynamism in performance of crowd platforms. Towards this end, we demonstrate CrowdUtility, a statistical model-ling based tool for evaluating multiple crowdsourcing plat-forms and recommending a platform that best suits the re-quirements of the requester. CrowdUtility uses an online Multi-Armed Bandit framework, to schedule tasks while op-timizing platform performance. We demonstrate an end-to end system starting from requirements specification, to plat-form recommendation, to real-time monitoring.