Integration of Vision Transformer Networks in YOLOv8 for Object Detection: Comparative Study on Plant Disease Detection

Abstract

Accurate detection of plant diseases is vital for global food security. While YOLOv8 represents a state-of-the-art detector, its convolutional backbone may lack a global contextual understanding of modern Vision Transformers (ViT). This paper presents a comprehensive study that integrates three ViT backbones—the Pyramid Vision Transformer (PVT), the Swin Transformer (SwinT), and the BiFormer—into the YOLOv8 architecture to benchmark their performance for plant disease detection. To ensure robust performance estimates, all models are trained three times on stratified dataset splits and evaluated on the PlantDoc dataset. While YOLOv8 is already a strong baseline for plant disease detection, our study demonstrates that integrating ViT backbones can elevate its performance substantially. The BiFormer-Small (BiFormer-S) yields the most significant improvement, reaching a mean mAP50-95 of 53.39% on the PlantDoc dataset. This represents a gain of 6.39 percentage points over its direct architectural baseline, YOLOv8s, which achieves 47.00% mAP50-95. Furthermore, it even surpasses the largest model in the family, YOLOv8x (49.22%), demonstrating a superior accuracy-efficiency trade-off. Beyond quantitative metrics, Explainable AI (XAI) analyses reveal that the BiFormer-S backbone produces sharper, more context-aware feature maps than the convolutional baseline, thereby improving the model’s ability to detect subtle or early-stage disease symptoms.