The influence of conversational agents’ role and communication style on user experience

Abstract

Conversational Agents (CAs) are characterized by their roles within a narrative and the communication style they adopt during conversations. Within computer games, users’ evaluation of the narrative is influenced by their estimation of CAs’ intelligence and believability. However, the impact of CAs’ roles and communication styles on users’ experience remains unclear. This research investigates such influence of CAs’ roles and communication styles through a crime-solving textual game. Four different CAs were developed and each of them was assigned to a role of either witness or suspect and to a communication style than can be either aggressive or cooperative. Communication styles were simulated through a Wizard of Oz method. Users’ task was to interact, through real-time written exchanges, with the four CAs and then to identify the culprit, assess the certainty of their judgments, and rank the CAs based on their conversational preferences. In addition, users’ experience was evaluated using perceptual measures (perceived intelligence and believability scales) and behavioral measures (including analysis of users’ input length, input delay, and conversation length). The results revealed that users’ evaluation of CAs’ intelligence and believability was primarily influenced by CAs’ roles. On the other hand, users’ conversational behaviors were mainly influenced by CAs’ communication styles. CAs’ communication styles also significantly determined users’ choice of the culprit and conversational preferences.