Split algorithms for sets of high-dimensional objects

Abstract

This study compares general, heuristic algorithms for splitting a set of high-dimensional objects. The algorithms aim at minimizing the overlap of the created subsets, while simultaneously satisfying a given balancing condition for their cardinalities. The intuitive goal is to enhance clustering and disjointness, and thereby to improve retrieval performance. Three categories of complexity are studied: O(N), 0(N log N), and 0(N'2)y when splitting a set of iV objects. A representative algorithm is suggested for each category, the main contribution being a new class of quadratic algorithms which take advantage of existing heuristics for the traveling salesman problem. All algorithms are quite general, making minimal assumptions about the underlying object domain. However, for low-dimensional (e.g. spatial) objects, specialized split techniques produce better results. The experiments are therefore restricted to splitting a set of signatures (bit-vectors in general). The trade-off between computation effort and result quality is clearly established.