Explaining Image Classifications with Near Misses, Near Hits and Prototypes: Supporting Domain Experts in Understanding Decision Boundaries

Abstract

We propose a method for explaining the results of black box image classifiers to domain experts and end users, combining two example-based explanatory approaches: Firstly, prototypes as representative data points for classes, and secondly, contrastive example comparisons in the form of near misses and near hits. A prototype globally explains the relevant characteristics for a entire class, whereas near hit and near miss explain the local decision boundary of a specific prediction. To combine both types of explanations within one framework is novel and we propose that presenting both types of explanations is especially helpful for domain experts in visual domains. To improve the faithfulness of the explanations, we investigated an unbiased, generic embedding and a model-related (model-specific) embedding for handling the images. The proposed approaches are evaluated regarding parameter selection and suitability on two different data sets – the well-known MNIST and a real-world industrial quality control data set. Finally, it is shown how global and local example-based explanation can be combined and realized within a demonstrator.