Effectiveness of Attention Mechanisms in YOLOv8 for Maritime Vessel Detection

Abstract

Maritime vessel detection in nearshore waters is a fundamental capability for artificial intelligence (AI)-enabled maritime transportation systems, including coastal monitoring, traffic management, and digital maritime services. Although attention mechanisms are widely incorporated into YOLO-based detectors, their relative effectiveness in marine environments under strictly controlled experimental conditions remains insufficiently clarified. This study presents a systematic comparison of Coordinate Attention (CA), Convolutional Block Attention Module (CBAM), and CLIP-based semantic fusion within a unified YOLOv8n framework for binary discrimination between ships and fishing boats in cluttered coastal imagery. All model variants were trained under identical data partitions and optimization settings to isolate architectural effects. The experimental results show that CA achieves the highest localization robustness (mAP@0.5:0.95 = 0.6127) and substantially improves precision (+7.13% over baseline), while CBAM provides the most balanced performance with the highest F1-score. In contrast, CLIP-based semantic fusion consistently degrades detection reliability, indicating limitations of global vision–language representations in small-scale maritime datasets. Precision–Recall and F1 analyses further reveal architecture-specific confidence calibration behaviors relevant to deployment-sensitive maritime applications. The findings provide practical guidance for selecting attention mechanisms in AI-driven maritime perception systems and support reliable AI integration in marine science and engineering applications.