κ NN algorithm with data-driven k value

Abstract

This paper proposes a new κ Nearest Neighbor (κNN) algorithmbased on sparse learning, so as to overcome the drawbacks of the previous κNN algorithm, such as the fixed κ value for each test sample and the neglect of the correlation of samples. Specifically, the paper reconstructs test samples by training samples to learn the optimal k value for each test sample, and then uses κNN algorithm with the learnt k value to conduct all kinds of tasks, such as classification, regression, and missing value imputation. The rationale of the proposed method is that different test samples should be assigned different κ values in κNN algorithm, and learning the optimal κ value for each test sample should be taken the correlation of data into account. To this end, in the reconstruction process, the proposed method is designed to achieve the minimal reconstruction error via a least square loss function, and employ an ℓ1-norm regularization term to create the element-wise sparsity in the reconstruction coefficient, i.e., sparsity appearing in the element of the coefficient matrix. For achieving effectiveness, the Locality Preserving Projection (LPP) is employed to keep the local structures of data. Finally, the experimental results on real datasets, and the experimental results show that the proposed κNN algorithm is better than the state-of-the-art algorithms in terms of different learning tasks, such as classification, regression, and missing value imputation.