Measuring Instance Hardness Using Data Complexity Measures

Abstract

Assessing the hardness of each instance in a problem is an important meta-knowledge which may leverage advances in Machine Learning. In classification problems, an instance can be regarded as difficult if it gets systematically misclassified by a diverse set of classification techniques with different biases. The instance hardness measures were proposed with the aim of relating data characteristics to this notion of intrinsic difficulty of the instances. There are also in the literature a large set of measures which are dedicated at describing the difficulty of a classification problem from a dataset-level perspective. In this paper these measures are decomposed at the instance-level, giving a perspective of how each individual example in a dataset contributes to its overall complexity. Experiments on synthetic and benchmark datasets demonstrate the proposed measures can provide a complementary instance hardness perspective when compared to those from related literature.