Domain of competency of classifiers on overlapping complexity of datasets using multi-label classification with meta-learning

Abstract

A classifier’s performance can be greatly influenced by the characteristics of the underlying dataset. We aim at investigating the connection between the overlapping complexity of dataset and the performance of a classifier in order to understand the domain of competence of these machine learning classifiers. In this paper, we report the results and implications of a study investigating the connection between four overlapping measures and the performance of three classifiers, namely KNN, C4.5 and SVM. In this study, we first evaluated the performance of the three classifiers over 1060 binary classification datasets. Next, we constructed a multi-label classification dataset by computing the four overlapping measures as features and multi-labeled with the competent classifiers over these 1060 binary classification datasets. The generated multi-label classification dataset is then used to estimate the domain of the competence of the three classifiers with respect to the overlapping complexity. This allowed us to express the domain of competence of these classifiers as a set of rules obtained through multi-label rule learning. We found classifiers’ performance invariably degraded with the datasets having high values of complexity measures (N1 and N3). This suggested for the existence of a strong negative correlation between the classifiers’ performance and class overlapping present in the data.