HMI with vision system to control manipulator by operator hand movement

Abstract

In the paper, the vision system for control of manipulator by operator's hand movement is presented. The Authors propose method which uses specific colors of markers. The fast camera 3iCube (frequency 50 Hz) was used and connected with PC by USB 3.0. The position of the manipulator was controlled by industrial controller. Connection between PLC and PC was realized by TCP/IP protocol. The two green markers were used to scale operator's working area. One green marker indicated a base coordinates of manipulator. The red marker was held in hand by operator. Movement of the red marker was in ratio 1:2 to manipulator. The research was focused on the correctness of detection of markers and response trajectory of the manipulator. Proposed method can be used to control manipulator movement by operator without uses classic control panel. The research proves possibility of application of the vision system for control of manipulator.