Relevance-Aware Content-based Image Retrieval using Deep Hybrid Feature Extraction

Abstract

Content-Based Image Retrieval (CBIR) requires balancing feature representation quality, computational efficiency, and robust performance across diverse image domains. Traditional methods lack semantic understanding, whereas deep learning approaches often exclude critical local structural information. This study presents a novel hybrid framework that effectively combines Histogram of Oriented Gradients (HOG) with EfficientNet through a two-stream architecture, enhanced by a query-sensitive co-attention mechanism and Fisher vector encoding. The framework employs an adaptive fusion strategy that dynamically adjusts feature contributions based on the query context and overcomes key limitations of existing approaches. Experimental evaluation on benchmark datasets demonstrates superior performance, achieving mean Average Precision scores of 0.89, 0.85, and 0.83 on Corel-1K, Oxford5K, and Paris6K datasets, respectively, representing a 3-5% improvement over state-of-the-art methods. The framework shows particular effectiveness in handling challenging scenarios such as viewpoint variations and partial occlusions, with landmark queries achieving a Precision@10 of 0.92. Comprehensive ablation studies validate the contribution of each component, with HOG feature integration and attention mechanism improving performance by 4.2% and 3.8%, respectively. The proposed approach successfully bridges the gap between traditional and deep learning methods while maintaining computational efficiency for practical applications.