Bringing information retrieval into crowdsourcing: A case study

Abstract

We propose a novel and economic framework for bringing information retrieval into crowdsourcing. Both crowdsourcing and information retrieval achieve mutual benefits, which result in (1) workers' quality control by using the query-oriented training; (2) cost savings in money and time; and (3) better qualified feedback information. In our case study, the costs of crowdsourcing for 18,260 jobs are as low as $47.25 and as short as 5 hours in total. Furthermore, the experimental results show that information retrieval techniques greatly reduce the workloads of crowdsourcing, which is only 5% of the original work. At the other hand, crowdsourcing improves the accuracy of the information retrieval system through providing qualified feedback information. © 2014 Springer International Publishing Switzerland.