Leveraging Human-Agent Collaboration for Multimodal Task Guidance with Concurrent Authoring Capabilities

Abstract

Instructive assistance systems are applied to assist users in performing a task by providing instructions. These systems are especially relevant for industrial workers and service technicians who must cope with the increasing task variety and complexity that resulted from the trend of applying cyber-physical production systems and small lot sizes. While current research mainly focuses on assembly work and the application of augmented reality, there is a lack of research for these systems for adapting to the challenging conditions of maintenance environments and for providing concurrent authoring capabilities. Within this PhD project, we aim to design a multimodal conversational agent as an instructive assistance system with authoring capabilities and evaluate the system in three different scenarios to close this research gap.