Confidence Sets for Fine-Grained Categorization and Plant Species Identification

Abstract

We present a new hierarchical strategy for fine-grained categorization. Standard, fully automated systems report a single estimate of the category, or perhaps a ranked list, but have non-neglible error rates for most realistic scenarios, which limits their utility. Instead, we propose a semi-automated system which outputs a it confidence set (CS)—a variable-length list of categories which contains the true one with high probability (e.g., a 99 % CS). Performance is then measured by the expected size of the CS, reflecting the effort required for final identification by the user. The implementation is based on a hierarchical clustering of the full set of categories. This tree of subsets provides a graded family of candidate CS’s containing visually similar categories. There is also a learned discriminant score for deciding between each subset and all others combined. Selection of the CS is based on the joint score likelihood under a Bayesian network model. We apply this method to determining the species of a plant from an image of a leaf against either a uniform or natural background. Extensive experiments are reported. We obtain superior results relative to existing methods for point estimates for scanned leaves and report the first useful results for natural images at the expense of asking the user to initialize the process by identifying specific landmarks.