Binary relevance multi-label conformal predictor

Abstract

The Conformal Prediction (CP) framework can be used for obtaining reliable confidence measures in Machine Learning applications. The confidence measures are guaranteed to be valid under the assumption that the data used are identically and independently distributed (i.i.d.). In this work, we extend the CP framework for multi-label classification, where an instance can belong to multiple classes in parallel. Applications include image tagging, document classification, and music classification. We give an overview of the Conformal Prediction framework, and we describe the developed Binary Relevance Multi-Label Conformal Predictor (BR-MLCP). We propose a new measure of confidence using Chebyshev’s inequality together with the hamming loss metric. Our experimental results demonstrate the reliability of our new confidence measure.