Multispectral imaging and deep learning for oil palm fruit bunch ripeness detection

Abstract

Oil palm fresh fruit bunches (FFBs) are the raw material of crude palm oil (CPO) on which ripeness levels of FFBs are essential to obtain good quality CPO. Most palm oil mills use experienced graders to evaluate FFB ripeness levels. Researchers have developed rapid and non-destructive methods for ripeness detection using computer vision (CV) and deep learning. However, most of the experiments used color cameras, such as a webcam or a smartphone, limited to visible wavelengths, and used still FFBs on–trees or on the ground. This study developed a light-emitting diode (LED)-based multispectral imaging system with deep learning for rapid and real-time ripeness detection of oil palm FFBs on a moving conveyor. The ripeness levels used were unripe and ripe. We also evaluated the spectrum of reflectance intensities for the ripeness levels. The ripeness detection system employed a two-class you only look once version 4 (YOLOv4) detection model using a dataset of 2000 annotated unripe and ripe FFB multispectral images and a video of 30 moving FFBs for real-time testing. The results show a promising method to detect oil palm FFB ripeness with an average accuracy of 99.66% and a speed range of 3.32-3.62 frame per second (FPS).