Virtual Triplets: Human-Agent Shared Control of Virtual Avatars

Abstract

We propose Virtual Triplets, a collaborative Virtual Reality system capable of human-human and human-agent interaction where users can choose to switch their control between two or more virtual avatars, while the virtual agent would take over the control of the free, unpossessed avatars. We developed a use-case scenario where a single human instructor concurrently supervises two students in a virtual classroom setting. Each student is assigned a personal instructor avatar where the instructor could switch the possession between these two avatars. When the human instructor attends to one of the students, the virtual agent controls the unpossessed avatar to assume the supervision role for another student. Virtual Triplets supports recording and playback of the user's actions. For example, the human instructor's demonstration shown to one student could be recorded and played back on a different avatar for another student. Moreover, the human instructor can choose to possess a top-down view instead of an individual avatar to have an overview of all the workspaces and give commands to the virtual instructor's avatar. Our goal is to enable parallelism in a collaborative environment to improve the user experience when multitasking is required for repetitive activities.