Design of a new classifier simulator

Abstract

Since standard data sets are not capable enough in evaluating classifier combination methods in multiple classifier systems, a new classifier simulator with sufficient diversity is proposed to generate artificial data sets. The simulator can generate simulating data for a problem of any number of classes and any classifier performance, and can also show pair wise dependency. It is achieved via a three-step algorithm: firstly building the confusion matrices of the classifiers on the basis of desired behavior, secondly generating the outputs of one classifier based on its confusion matrix, and then producing the outputs of other classifiers. The detailed generating algorithm is discussed. Experiments on majority voting combination method shows that negative correlation could improve the accuracy of multiple classifier systems, which indicates the validity of the proposed simulator. © Springer-Verlag Berlin Heidelberg 2005.