Self-selection bias in reputation systems

Abstract

Reputation systems appear to be inherently biased towards better-than-average ratings. We explain this as a consequence of self-selection, where reviewers are drawn disproportionately from the subset of potential consumers favorably predisposed toward the resource. Inflated ratings tend to attract consumers with lower expected value, who have a greater chance of disappointment. Paradoxically, the more accurate the ratings, the greater the degree of self-selection, and the faster the ratings become biased. We derive sufficient conditions under which biased ratings occur. Finally, we outline a potential solution to this problem that involves stating expectations before interaction with the resource, and expressing subsequent ratings in terms of delight or disappointment. © 2007 International Federation for Information Processing.