Weighted sampling without replacement

Abstract

Comparing concentration properties of uniform sampling with and without replacement has a long history which can be traced back to the pioneer work of Hoeffding (1963). The goal of this note is to extend this comparison to the case of non-uniform weights, using a coupling between samples drawn with and without replacement. When the items’ weights are arranged in the same order as their values, we show that the induced coupling for the cumulative values is a submartingale coupling. As a consequence, the powerful Chernoff-type upper-tail estimates known for sampling with replacement automatically transfer to the case of sampling without replacement. For general weights, we use the same coupling to establish a sub-Gaussian concentration inequality. As the sample size approaches the total number of items, the variance factor in this inequality displays the same kind of sharpening as Serfling (1974) identified in the case of uniform weights. We also construct an other martingale coupling which allows us to answer a question raised by Luh and Pippenger (2014) on sampling in Polya urns with different replacement numbers.