Integrating data- and model-driven analysis of RGB-D images

Abstract

There is a growing use of RGB-D sensors in vision-based robot perception. A reliable 3D object recognition requires the integration of image-driven and model-based analysis. Only then the low-level image-like representation can be successfully transformed into a symbolic description with equivalent semantics, considered by the ontology-level representation of an autonomous robot system. An RGB-D image analysis approach is proposed that consists of a data-driven hypothesis generation step and a generic model-based object recognition step. Initially point clusters are created assuming to represent 3D object hypotheses. In parallel, 3D surface patches are estimated, 2D image textures and shapes are classified, building multi-modal image segmentation data. In the model-driven step, a built-in knowledge about basic solids, shapes and textures is used to verify the point clusters in terms of meaningful volume-like aggregates, and to create (or to recognize) generic 3D object models.