Real-time crop recognition in transplanted fields with prominent weed growth: A visual-attention-based approach

Abstract

Crop recognition is one of the key processes for robotic weeding in precision agriculture, which remains an open problem due to the unstructured field environment and the wide variety of plant species. It becomes especially challenging when the weeds are prominent and overlap with the crop plants. This paper presents a novel method for recognizing crop plants of field images with a high weed presence. This method segments crop plants from overlapped weeds based on the visual attention mechanism of the human visual system using a convolutional neural network. The network utilizes ResNet-10 as backbone, while introducing side outputs and short connections for multi-scale feature fusion. The Adaptive Affinity Fields method is adopted to improve the segmentation at object boundaries and for fine structures. To train and test the network, a field image dataset has been created which consists of 788 color images with manually segmented annotations. The images are captured under challenging conditions with extremely high weed pressure. The experimental results show that the proposed method can accurately segment crops from weeds and soil, with mean absolute errors less than 0.005 and F-measure scores exceeding 97%. In terms of efficiency, the proposed method can process up to 169 images per second when accelerated by a NVIDIA RTX 2080Ti graphics processing unit (GPU), and operate at approximately 5.6 Hz in a Jetson TX2 embedded computer. The results indicate that the proposed method has the potential to provide an efficient solution for recognizing crop plants, even in the presence of severe weed growth. The code and the dataset are available at https://github.com/ZhangXG001/Real-Time-Crop-Recognition.