Multimodal Intention Recognition Combining Head Motion and Throat Vibration for Underwater Superlimbs

Abstract

This paper presents a novel solution for underwater intention recognition that simultaneously detects head motion and throat vibration, enhancing multimodal human-robot interactions for underwater diving. The system pairs with an underwater supernumerary robotic limb (SuperLimb), providing propulsion assistance to reduce the diver’s physical load and mental fatigue. An inertial measurement unit monitors head motion, while a throat microphone captures vocal vibrations. Learning algorithms process these signals to accurately interpret the diver’s intentions and map them to the SuperLimb for posture management. The system features a compact design optimized for diving scenarios and includes a multimodal, real-time classification algorithm to distinguish various head motions and vocal signals. By collecting and analyzing underwater throat vibration data, the study demonstrates the feasibility of this approach, enabling continuous motion commands for enhanced diving assistance. The results show that the head motion recognition component of the system achieved a high classification accuracy of 95%, and throat vibration classification reached 86% accuracy on land and 89% underwater for various purposes. Note to Practitioners—This paper tackles the “diver’s wearable dilemma,” where effective communication and underwater robot control clash with traditional diver wearables’ constraints. Current methods, like hand gestures for interacting with underwater robots, disrupt natural limb movement and increase diver fatigue. To address this, we introduce a novel underwater intention recognition system that uses head movements and throat vibrations detected by an Inertial Measurement Unit (IMU) and a throat microphone. This hands-free interaction enhances wearable diving aids by reducing physical strain. We detail the algorithm for classifying and mapping these inputs to robot control commands, enabling real-time, continuous robot operation. Our approach significantly advances healthcare automation in commercial diving, improving diver safety and efficiency. Preliminary experiments show promise, but further field testing is needed to validate effectiveness in real-world conditions and ensure system reliability under extreme environments to prevent accidents from failures or interaction errors. Based on a user study conducted in Virtual Reality, the proposed interaction method maintains a consistent mental workload for users across tasks of varying difficulty and does not interfere with other human functions.