Multimodal Radar-Vision for Oil Palm Fresh Fruit Bunch Ripeness Classification

Abstract

This work presents a multimodal radar-vision framework for non-destructive ripeness classification of oil palm fresh fruit bunches (FFBs) under field conditions. A continuous wave 2.4 GHz radar and a digital scale capture backscattered power and bunch mass as internal cues related to dielectric properties and growth, while RGB images provide external color and texture information. Radar-weight features (power, weight, and power-to-weight ratio) are modeled using an extreme gradient boosting (XGBoost) classifier, and a YOLOv8-Nano detector is trained using an expert-annotated RGB dataset to localize and classify bunches into four maturity stages (Unripe, Underripe, Ripe, Overripe). Decision-level late fusion combines the class-probability outputs of both branches through mean, weighted, and stacking schemes. Experiments on radar-weight data (255 samples per class) acquired at a commercial loading ramp show that the unimodal radar-weight and vision models reach 89.9% and 94.3% accuracy, respectively, with YOLOv8-Nano achieving mAP@0.5 of 0.99. The best fusion configuration, based on stacking, attains 96.9% accuracy and 97.3% F1-macro, demonstrating that radar-weight information provides complementary evidence that stabilizes vision-based grading and supports more consistent harvest decisions.