Enhancing Object Detection Robustness in Industrial UAVs Through Multimodal Deep Feature Fusion

Abstract

This paper presents a novel framework for enhancing fire detection robustness in industrial settings by integrating multimodal deep feature fusion with uncrewed aerial vehicle (UAV)-assisted thermal and visual inspection. The system employs drones equipped with multimodal sensorsałsuch as high-resolution visual, thermal, and hyperspectral camerasałto capture complementary data streams, simulating a comprehensive perceptual system to detect and evaluate potential fire incidents, including nascent smoldering, open flames, and thermal anomalies. At the core of the methodology is a deep fusion model that emulates human multi-sensory integration, enabling adaptive focus on critical thermal and visual cues under complex industrial conditions like smoke, steam, and glare. A key component is a multi-scale feature extraction pipeline that processes inputs from each modality to efficiently identify regions of interest with varying thermal and visual signatures. To create a robust and discriminative feature representation, deep features from convolutional neural networks (CNNs) are fused using attention-based mechanisms, which prioritize the most informative modalities and spatial regions for a given scenario. This ensures robust handling of environmental noise and variability typical of industrial plants. An intelligent region prioritization algorithm is introduced to minimize computational cost while maximizing detection accuracy by focusing processing resources on high-risk areas. The system further employs a context-aware inspection path to streamline data acquisition, dynamically adjusting the UAV’s flight pattern based on the fused sensor data to investigate potential hazards. A deep learning-based inference system, combining the fused features for joint analysis, classifies threats based on intensity, growth rate, and contextual risk to infrastructure. Validation experiments conducted on a dedicated industrial fire dataset and several public benchmarks demonstrate the system’s superiority in identifying subtle thermal anomalies and reducing false alarms by effectively capturing the complex interactions between multiple sensory inputs. The proposed approach significantly improves the automation, accuracy, and robustness of fire detection for industrial UAVs, contributing to safer and more proactive facility monitoring practices.