A Study on the Benefits and Drawbacks of Adaptivity in AI-generated Explanations

Abstract

It is commonly assumed that explanations should be tailored to the addressee in order to yield higher understanding. Consequently, much work on explainable intelligent agents has been directed to user-adapted explanations. However, recent studies show ambiguous results with regard to the efficiency of adaptive and non-adaptive explanations. This raises the question whether an explanation, generated by a socially interactive agent, should be adapted. In this paper, we present a general approach to adaptive explanation generation as a non-stationary decision process, and we study the benefits and pitfalls of adapting explanations in an ongoing interaction with a user. Specifically, we report results from a between-subject online evaluation in a game explanation domain with three conditions (non-interactive, interactive but non-adaptive, adaptive). Results show that the decision for or against adaptivity depends on the goal of the explanation, the complexity of the domain and external constraints. Based on the collected data we discuss challenges that arise from the individuality of adaptive dialogues, such as comparability and the tendency to produce results with a large variance.