Encoding High-Order Statistics in VLAD for Scalable Image Retrieval

Abstract

We revisit the implicit design choices in the popular vector of locally aggregated descriptors (VLAD), which aggregates the residuals of local image descriptors. Since original VLAD ignores high-order statistics the resultant vector is not discriminative enough. We address this issue by exploiting high-order statistics for gaining complementary information. Our contributions are two-fold: First, we present a novel high-order VLAD (HO-VLAD) with increased discriminative power. Next, we propose a light-weight retrieval framework to demonstrate HO-VLAD’s effectiveness for scalable image retrieval. Systematic experiments on two challenging public databases (INRIA Holidays, UKBench) exhibit a consistent improvement of performance with limited computational costs.