Prototype Network for Predicting Occluded Picking Position Based on Lychee Phenotypic Features

Abstract

The automated harvesting of clustered fruits relies on fast and accurate visual perception. However, the obscured stem diameters via leaf occlusion lack any discernible texture patterns. Nevertheless, our human visual system can often judge the position of harvesting points. Inspired by this, the aim of this paper is to address this issue by leveraging the morphology and the distribution of fruit contour gradient directions. Firstly, this paper proposes the calculation of fruit normal vectors using edge computation and gradient direction distribution. The research results demonstrate a significant mathematical relationship between the contour edge gradient and its inclination angle, but the experiments show that the standard error projected onto the Y-axis is smaller, which is evidently more conducive to distinguishing the gradient distribution. Secondly, for the front view of occluded lychee clusters, a fully convolutional, feature prototype-based one-stage instance segmentation network is proposed, named the lychee picking point prediction network (LP3Net). This network can achieve high accuracy and real-time instance segmentation, as well as for occluded and overlapping fruits. Finally, the experimental results show that the LP3Net based on this study, along with lychee phenotypic features, achieves an average location accuracy reaching 82%, significantly improving the precision of harvesting point localization for lychee clusters.