Qualitative robustness of bootstrap approximations for kernel based methods

Abstract

The finite sample distribution of many nonparametric methods from statistical learning theory is unknown because the distribution P from which the data were generated is unknown and because there often exist only asymptotical results on the behaviour of such methods. The goal of this contribution is to show that bootstrap approximations of an estimator which is based on a continuous operator from the set of Borel probability distributions defined on a compact metric space into a complete separable metric space is stable in the sense of qualitative robustness. As a special case it is shown that, under certain regularity conditions, bootstrap approximations for many (general) support vector machines (SVM) are qualitatively robust, both for the real-valued SVM risk and for the SVM itself. The required regularity conditions involve the loss function and the kernel, but not the unknown distribution P. Hence, these conditions are verifiable in advance and are not data dependent.