"When Can I Trust It?" Contextualising Explainability Methods for Classifiers

Abstract

The need for artificial intelligence systems to expose reasons for promoted decisions grows with the prevalence of these systems in society. In this work, we study, for carefully selected images, how an end user's trust is affected by visual explanations. Additionally, we complement our work by probing the pretrained neural network's consistency for the selected images. Our research approach exposes the brittleness in these systems pointing toward a need to develop benchmarking methods connecting visual explanations to training data distribution and, additionally, move away from a flat output hierarchy toward including a concept ontology that matches the target domain. Additional material and code to reproduce experiments can be found at https://github.com/k3larra/IKR.