SVM with random labels

Abstract

We devise an SVM for partitioning a sample space affected by random binary labels. In the hypothesis that a smooth, possibly symmetric, conditional label distribution graduates the passage from the all 0-label domain to the all 1-label domain and under other regularity conditions, the algorithm supplies an estimate of the above probabilities. Within the Algorithmic Inference framework, the randomness of the labels maintains the main features of the binary classification problem, yet adding a further dimension to the search space. Namely the new dimension of each point in the original space hosts the uniform seeds accounting for the randomness of the labels, so that the problem becomes that of separating the points in the augmented space. We solve it with a new kind of bootstrap technique. As for error bounds of the proposed algorithm, we obtain confidence intervals that are up to an order narrower than those supplied in the literature. This benefit comes from the fact that: (i) we devise a special algorithm to take into account the random profile of the labels; (ii) we know the number of support vectors really employed, as an ancillary output of the learning procedure; and (iii) we can appreciate confidence intervals of misclassifying probability exactly in function of the cardinality of these vectors. We numerically check these results by measuring the coverage of the confidence intervals. © Springer-Verlag Berlin Heidelberg 2007.