Indoor RGB-D object detection with the guidance of hand-held objects

Abstract

In this paper, towards intelligent learning like human for robots, we propose a novel framework to detect indoor objects with the guidance of hand-held objects. In this framework, firstly, we leverage the segmentation algorithm to obtain hand-held objects proposals, and propose another segmentation algorithm based on depth information to obtain object proposals in the indoor scene. Second, to solve the problem of the diversity of data distribution between hand-held objects and indoor objects, we employ the unsupervised adaptive deep neural networks to learn adaptive features on the RGB and depth modality. To evaluate the proposed framework, we conduct experiments on the HOD-16 dataset and indoor scene dataset. The experimental results demonstrate that the proposed framework improves the performance of the indoor object detection in RGB, depth and fused modality respectively.