Exploring user profiles based on their explainability requirements in interactive systems

Abstract

Depending on the way they are designed, interactive systems may inadvertently influence the opinions, choices and actions of their users. To avoid this problem, there has been a growing demand for this type of system to be able to explain to users how their outputs are generated. This demand is called the "right to an explanation", or the explainability requirement. User perceptions are still little addressed in this context, for example: (1) do people value software explainability? (2) what are the typical user profiles in terms of their demand for explainability? This work seeks to elucidate users' perception and to identify profiles of users in terms of their perceptions of explainability requirement. The method consists of a questionnaire, data mining algorithms and statistical analysis to uncover and analyze the profiles of users. In a case study conducted with 61 people, we found 6 profiles, each one representing a different perception and type of requirement about software explainability. The implications of these profiles for the design of interactive systems are discussed.