Reducing hubness for kernel regression

Abstract

In this paper, we point out that hubness—some samples in a high-dimensional dataset emerge as hubs that are similar to many other samples—influences the performance of kernel regression. Because the dimension of feature spaces induced by kernels is usually very high, hubness occurs, giving rise to the problem of multicollinearity, which is known as a cause of instability of regression results. We propose hubnessreduced kernels for kernel regression as an extension of a previous approach for kNN classification that reduces spatial centrality to eliminate hubness.