Human and robotic fish interaction controlled using hand gesture image processing

Abstract

This paper is about the control of robotic fish movement in an aquarium via human hand gestures detected by image sensors attached in the aquarium. In this study, sensors actively interact with humans and robotic fish. Image and radio frequency sensors are used to identify the position and color of robotic fish. Recently, we have studied human interactive control based on hand gesture recognition. Image sensors send the input signals of hand gestures obtained from real-time video images processed using tracking control algorithms, such as color mark, stop zone, and lead-lag tracking algorithms, to robotic fish. The movement of robotic fish is controlled via the movement of the two hands, where the left hand is for the fish to be controlled and the right hand is for controlling the movement of the robotic fish. Hand gesture recognition consists of hand feature segmentation and gesture recognition from the hand features. Our results show that interactive human control using hand gestures successfully controls the movement of robotic fish.