Large scale instance selection by means of a parallel algorithm

Abstract

Instance selection is becoming more and more relevant due to the huge amount of data that is constantly being produced. However, although current algorithms are useful for fairly large datasets, many scaling problems are found when the number of instances is of hundred of thousands or millions. Most instance selection algorithms are of complexity at least O(n 2), n being the number of instances. When we face huge problems, the scalability becomes an issue, and most of the algorithms are not applicable. This paper presents a way of removing this difficulty by means of a parallel algorithm that performs several rounds of instance selection on subsets of the original dataset. These rounds are combined using a voting scheme to allow a very good performance in terms of testing error and storage reduction, while the execution time of the process is decreased very significantly. The method is specially efficient when we use instance selection algorithms that are of a high computational cost. An extensive comparison in 35 datasets of medium and large sizes from the UCI Machine Learning Repository shows the usefulness of our method. Additionally, the method is applied to 6 huge datasets (from three hundred thousands to more than four millions instances) with very good results and fast execution time. © 2010 Springer-Verlag Berlin Heidelberg.