Are performance differences of interest operators statistically significant?

Abstract

The differences in performance of a range of interest operators are examined in a null hypothesis framework using McNemar's test on a widely-used database of images, to ascertain whether these apparent differences are statistically significant. It is found that some performance differences are indeed statistically significant, though most of them are at a fairly low level of confidence, i.e. with about a 1-in-20 chance that the results could be due to features of the evaluation database. A new evaluation measure i.e. accurate homography estimation is used to characterize the performance of feature extraction algorithms.Results suggest that operators employing longer descriptors are more reliable. © 2011 Springer-Verlag.