Optimal Truth-Tracking Rules for the Aggregation of Incomplete Judgments

Abstract

Suppose you need to determine the correct answer to a complex question that depends on two logically independent premises. You can ask several agents to each evaluate either just one of those premises (which they can do with relatively high accuracy) or both premises (in which case their need to multitask will lower their individual accuracy). We first determine the optimal rule to aggregate the individual judgments reported by the agents and then analyse their strategic incentives, depending on whether they are motivated by (i) the group tracking the truth, by (ii) maximising their own reputation, or by (iii) maximising the agreement of the group’s findings with their own private judgments. We also study the problem of deciding how many agents to ask for two judgments and how many to ask for just a single judgment.