Adaptive quantization of the high-dimensional data for efficient KNN processing

Abstract

In this paper, we present a novel index structure, called the SA-tree, to speed up processing of high-dimensional K-nearest neighbor (KNN) queries. The SA-tree employs data clustering and compression, i.e. utilizes the characteristics of each cluster to adaptively compress feature vectors into bit-strings. Hence our proposed mechanism can reduce the disk I/O and computational cost significantly, and adapt to different data distributions. We also develop efficient KNN search algorithms using MinMax Pruning and Partial MinDist Pruning methods. We conducted extensive experiments to evaluate the SA-tree and the results show that our approaches provide superior performance. © Springer-Verlag 2004.