Segmentation of Plant Point Cloud based on Deep Learning Method

Abstract

The acquisition of plant phenotypic parameters from three-dimensional (3D) point cloud often requires a certain amount of manual intervention and manual setting of thresholds, which brings limitations to the automatic high-throughput plant phenotyping. This paper introduces a deep learning-based method for segmenting plant parts from 3D point cloud. The training network used in our approach is a point cloud instance segmentation network based on 3D bounding box regression called 3D-BoNet. This network consists of a backbone and two branch network: (1)The backbone network is used for extracting the local feature and global feature of point cloud. (2)Two branch networks include the bounding box prediction network and the point-level mask prediction network. The instance bounding box of point cloud is predicted through the bounding box prediction network by inputting point cloud feature into the association layer and multi-criteria loss function, and the instance point-level mask is predicted through the point-level mask prediction network. Finally, our method combines the network, point cloud reconstruction method, data annotation, and data augmentation to complete specific segmentation task and the final segmentation results are evaluated. The experimental results show that the augmentation dataset based on rotation is effective for leaf segmentation, and the improvement of the backbone network also shows a significant effect on the instance segmentation network. For the individual leaf segmentation task, the best mean accuracy is 0.809, and the mean recall rate is 0.884; while for the individual plant segmentation task, the best mean accuracy is 0.875, the mean recall rate is 0.897.