Fruit recognition method for a harvesting robot with RGB-D cameras

Abstract

In this study, we present a recognition method for a fruit-harvesting robot to automate the harvesting of pears and apples on joint V-shaped trellis. It is necessary to recognize the three-dimensional position of the harvesting target for harvesting by the fruit-harvesting robot to insert its end-effector. However, the RGB-D (red, green, blue and depth) camera on the harvesting robot has a problem in that the point cloud obtained in outdoor environments can be inaccurate. Therefore, in this study, we propose an effective method for the harvesting robot to recognize fruits using not only three-dimensional information obtained from the RGB-D camera but also two-dimensional images and information from the camera. Furthermore, we report a method for determining the ripeness of pears using the information on fruit detection. Through experiments, we confirmed that the proposed method satisfies the accuracy required for a harvesting robot to continuously harvest fruits.