Which image pairs will cosegment well? Predicting partners for cosegmentation

Abstract

Cosegmentation methods segment multiple related images jointly, exploiting their shared appearance to generate more robust foreground models. While existing approaches assume that an oracle will specify which pairs of images are amenable to cosegmentation, in many scenarios such external information may be difficult to obtain. This is problematic, since coupling the “wrong” images for segmentation—even images of the same object class—can actually deteriorate performance relative to single-image segmentation. Rather than manually specify partner images for cosegmentation, we propose to automatically predict which images will cosegment well together. We develop a learning-to-rank approach that identifies good partners, based on paired descriptors capturing the images’ amenability to joint segmentation.We compare our approach to alternative methods for partnering images, including basic image similarity, and show the advantages on two challenging datasets.