Selecting and combining classifiers simultaneously with particle swarm optimization

Abstract

A weighted combination model of multiple classifier systems based on Particle Swarm Optimization was reviewed, which took sum rule and majority vote as special cases. It was observed that the rejection of weak classifier in the combination model can improve classification performance. Inspired by this observation, we presented a problem that how to choose the useful classifiers in a given ensemble, especially in the reviewed model. In this study, a combination algorithm was proposed, which implemented classifiers' selection and combination simultaneously with particle swarm optimization. We describe the implementation details, including particles encoding and fitness evaluation. Nine data sets from UCI Machine Learning Repository were used in the experiment to justify the validity of the method. Experimental results show that the propose model obtained the best performance on 5 out of 9 data sets, and averagely outperforms the reviewed model, majority voting, max rule, min rule, mean rule, median rule and product rule. The results were analysed from the point of the characteristic of data set. © 2009 Asian Network for Scientific Information.