Simulating gamified crowdsourcing of knowledge base refinement: Effects of game rule design

Abstract

This paper discusses a gamification design for knowledge base refinement. We present a simulation model for game playing and examine the possibility of estimating the effects of game rule design. The maintenance of a knowledge base involves human intervention and is one of the application domains of human computation. Using the concept of crowdsourcing, refinement tasks can be delegated to many casual users over a network. In addition, gamification, such as games with a purpose (GWAP), is a useful idea to motivate workers in crowdsourcing by making tasks enjoyable. For effective gamification, designing the game rules is critical. In this paper, we present a model for simulating the gamified knowledge base refinement process to estimate the effects of different game rule designs beforehand. We use a rental apartment FAQ system as an example knowledge base and show that by invoking game playing when a problem is found in a knowledge base, it can be properly updated. In addition, we discuss the effectiveness of using a calibration quiz to estimate the reliability of users.