Balancing automation and control: user perceptions of tool invocations in conversational agents

Abstract

: Advances in large language models (LLMs) have enabled conversational agents (CAs) to invoke external functions dynamically, yet little is known about how different tool-calling (TC) strategies affect user experience. This study compares two approaches – automatic execution based on conversational context versus user-confirmed execution – using a German-language CA in a business-travel scenario. In a between-subjects experiment, 451 employed adult participants (aged 18–65) were randomly assigned to one of the two TC conditions, completed a structured travel request interaction, and then rated their perceived trust (PTru), autonomy (PA), transparency (PTrn), ease of use (PEU), and usefulness (PU). Our findings indicate that TC strategy had a significant overall impact on users’ combined perceptions, and that demographic factors – particularly age – played a critical role in shaping these responses, highlighting the need for adaptive, inclusive design practices that balance automation with user control. CCS Concepts ∙ Human-centered computing → Human computer interaction (HCI) → Interactive systems and tools ∙ Human-centered computing → Interaction design → Systems and tools for interaction design.