Inference for the top-k rank list problem

Abstract

Consider a problem where N items (objects or individuals) are judged by assessors using their perceptions of a set of performance criteria, or alternatively by technical devices. In particular, two assessors might rank the items between 1 and N on the basis of relative performance, independently of each other. We aggregate the rank lists in that we assign one if the two assessors agree, and zero otherwise. How far can we continue into this sequence of 0's and 1's before randomness takes over? In this paper we suggest methods and algorithms for addressing this problem.