Multi-Modal Weights Sharing and Hierarchical Feature Fusion for RGBD Salient Object Detection

Abstract

Salient object detection (SOD) aims to identify and locate the most attractive regions in an image, which has been widely used in various vision tasks. Recent years, with the development of RGBD sensor technology, depth information of scenes becomes available for image understanding. In this paper, we systematically investigate and evaluate on how to integrate depth cues in a pre-trained deep network and learn informative features for SOD. First, we propose a CNN-based cross-modal transfer learning, which learn knowledge from sufficient labeled RGB salient object datasets and guide the depth domain feature extraction. Then we design a feature fusion module to fuse the complementary features in a hierarchical manner. At last, the final saliency map is obtained by integrating multi-scale information step by step. Extensive experiments on five popular RGBD benchmark datasets demonstrate that our proposed approach achieves significant improvements and outperforms the state-of-the-art methods.