Aggregating crowd opinions using shapley value regression

Abstract

Crowdsourcing is becoming increasingly popular in various tasks. Aggregating answers from workers in crowdsouring has been a widely used technique for providing many applications and services. To aggregate these answers, fair evaluation of workers is important to motivate them to give high quality answers. However, it is difficult to fairly evaluate workers if their answers show a high degree of correlation. In this paper, we propose to use the Shapley value regression as a means to address this problem. The regression technique is based on ideas developed from cooperative game theory to evaluate the relative importance of explanatory variables in reducing the error. We also exploit sparseness of worker collaboration graph to effectively calculate the Shapley value, since it requires an exponential computation time to calculate the Shapley value.