Skin-interfaced multimodal sensing and tactile feedback system as enhanced human-machine interface for closed-loop drone control

Abstract

Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone’s posture into two-dimensional tactile information, enhancing the operator’s perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation–based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone’s posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design.