Multi-Label Confusion Tensor

Abstract

The confusion matrix is the tool commonly used for the evaluation of the performance of a classification algorithm. While the computation of the confusion matrix for multi-class classification follows a well-developed procedure, the common approach for computing the confusion matrix for multi-label classification suffers from the ambiguity related to one-vs-rest strategy and ignores the possibility that predictions could be partially correct, which also leads to inaccuracies of the derived evaluation metric. Only recently, the two approaches dealing with the computation of multi-label confusion matrix have been proposed which take into account the specifics of multi-label classification. In this work, a new method for calculating evaluation metrics for multi-label classification is proposed. The proposed method is based on the calculation of two confusion matrices combined into the confusion tensor. It builds upon the insights into the shortcomings of the two existing approaches for calculating the multi-label confusion matrix. The main drawback of these techniques is their inability to compute precision and recall precisely. The Multi-Label Confusion Tensor was tested on synthetic and real data and compared with existing methods for calculating the multi-label confusion matrix. The source code and the data used to test the methodology are made publicly available.