Predicting Mid Air Interaction Movement sand Fatigue Using Deep Rein force ment Learning

Abstract

A common problem of mid-air interaction is excessive arm fatigue, known as the "Gorilla arm"effect. To predict and pre vent such problems at a low cost, we investigate user testing of mid-air interaction without real users, utilizing biomechani- cally simulated AI agents trained using deep Reinforcement Learning (RL). We implement this in a pointing task and four experimental conditions, demonstrating that the simulated fa tigue data matches human fatigue data. We also compare two effort models: 1) instantaneous joint torques commonly used in computer animation and robotics, and 2) the recent Three Compartment Controller (3CC-r) model from biomechanical literature. 3CC-r yields movements that are both more effi cient and relaxed, whereas with instantaneous joint torques, the RL agent can easily generate movements that are quickly tiring or only reach the targets slowly and inaccurately. Our work demonstrates that deep RL combined with the 3CC-/-pro vides a viable tool for predicting both interaction movements and user experience in silico, without users.