Adaptive budget allocation for sequential tasks in crowdsourcing

Abstract

This paper proposes a new budget allocation method for crowdsourced sequential tasks. Sequential tasks mean that an output of a task becomes an input to another task, and the quality of the final artifact depends on the qualities of the preceding tasks. In crowdsourcing, the abilities of workers are often difficult to learn in advance. Thus, the fixed budget allocation to the component tasks cannot respond to the realized situation. Also, the requester is often difficult to evaluate the quality of intermediate artifacts accurately, which results in misallocating the budget and wasting a budget. To overcome these difficulties, we have developed a contingent budget allocation method, i.e., generating a conditional plan given uncertainty about the intermediate states and action effects, by formalized a problem as POMDP and introducing a quality evaluation action. The experimental results show that the proposed method can find a solution in a reasonable time and improve the quality of the final artifact.